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ABSTRACT 


We  examine  the  problem  of  i gambler  desiring  to  maximize  the 
expected  value  of  a concave  increasing  utility  function  after 
a fixed,  finite  number  of  plays  against  a casino  which  offers 
favourable  bets  (in  the  sense  of  positive  expected  gains) . 

The  gambles  are  assumed  to  have  any  distribution,  which  may 
change  from  play  to  play  according  to  a Markov  chain,  but  is 
known  to  the  player  previous  to  his  placing  each  wager. 

Although  the  optimal  wager  does  not  exhibit  simple  monotonicity 
properties  as  a function  of  the  gambler's  current  fortune,  we 
exhibit  two  linear  functions  of  the  wager  and  fortune  that  are 
indeed  monotone  and  allow  to  bound  above  and  below  the  optimal 
wager  for  any  fortune,  given  that  for  any  other  fortune.  These 
functions  generalize  the  statements  "the  richer  you  are  the  more 
you  save"  and  "the  richer  you  are  the  more  you  should  try  to 
get." 

The  dependence  of  the  optimal  bet  on  the  gamble  currently  avail- 
able is  examined,  and,  in  the  case  of  coin  tossing  games,  it  is 
shown  that  "the  better  the  gamble,  the  more  you  should  bet." 

The  case  in  which  the  distribution  of  the  gamble  is  unknown, 
except  for  a prior  distribution  over  a set  of  possible  alterna- 
tives is  also  analyzed  and  results  similar  to  the  above  are 
exhibited. 
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CHAPTER  0 


INTRODUCTION 

This  thesis  is  concerned  with  the  study  of  optimal  gambling 
systems  in  favourable  games  with  finite  horizon,  under  the  assumption 
that  the  gambler's  utility  depends  only  on  his  final  fortune. 

Historically  it  has  been  assumed  that  any  gambling  situation  a 
player  enters  must  be  unfavourable  to  him.  During  the  past  fifteen 
years  however,  the  existence  of  favourable  or  super fair  games  has  been 
identified  by  Thorp,  Epstein,  Kassouf,  and  Fisher  and  Lorie  among 
others.  Such  gambles  as  blackjack,  baccarat,  stocks  and  options  have 
been  shown  to  be  superfair.  A number  of  interesting  papers  on  favour- 
able games  have  been  written,  and  some  statistical  work  has  been 
done  in  attempting  to  identify  more  superfair  gambles. 

We  consider  models  in  which  the  outcome  of  each  bet  is  determined 
by  a random  variable,  whose  distribution  depends  on  some  "state  of  the 
universe."  This  state  of  the  universe,  although  known  to  the  gambler, 
may  change  from  play  to  play  in  a random  way.  The  player  must  choose 
his  wager  at  each  stage  of  the  game,  and  is  assumed  to  act  with  the 
objective  of  maximizing  his  utility  at  the  end  of  the  game.  We  also 
consider  the  case  where  the  state  of  the  universe  is  known  to  the 
gambler  only  in  distribution,  allowing  the  gambler  to  incorporate  the 
information  obtained  as  the  outcome  of  each  gamble  becomes  known. 

Preliminaries 

Probability  theory  was  born  out  of  the  efforts  of  early  mathemati- 
cians to  study  gambling  problems.  Epstein  [6]  gives  an  interesting 
review  of  the  history  of  probability  and  its  relation  to  gambling. 
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The  past  twenty  years  have  seen  a rebirth  of  interest  in  gambling 
theory.  Dubins*  and  Savage's  work  "How  to  Gamble  if  You  Must"  [5] 
considers  unfavourable  gambling  situations  and  the  optimal  strategies 
for  these  games.  They  prove,  among  other  things  that  if  the  gambler's 
goal  is  to  reach  a fortune  of  N before  going  broke,  and  he  is  allowed 
even  money  wagers  with  probability  p < h of  success,  then  bold  play 
(i.e.  bet  min  {N  - i,i}  if  your  fortune  is  i)  is  optimal.  Ross  [16] 
also  considers  subfair  coin  tossing  games  and  shows  that  playing  timid- 
ly (i.e.  betting  the  minimum)  maximizes  the  expected  playing  time 
until  the  gambler  goes  broke. 

The  first  modern  day  author  to  consider  superfair  games  is  Kelly 
[11].  He  relates  gambling  to  information  theory  by  considering  a 
gambler  receiving  information  on  the  outcome  of  an  event  through  a 
noisy  channel,  and  being  able  to  bet  on  the  event  at  the  original 
odds.  For  a binary  channel  with  probability  p of  correct  trans- 
mission, Kelly  shows  that  if  the  gambler  wishes  to  maximize  lim  — LnS 

n-*» 

(where  S is  the  gambler's  fortune  after  n gambles)  and  he  is 
n 

restricted  to  betting  a constant  proportion  of  his  current  fortune  at 
each  time  period,  then  he  should  bet  the  proportion  (2p  - 1)  . 

Bellman  and  Kalaba  [1],  using  Kelly's  model,  show  that  betting  the 
proportion  (2p  - 1)  maximizes  ELnS^  for  all  n . 

Ross  in  [16]  also  considers  fair  and  superfair  coin  tossing  games 
and  shows  that  timid  play  maximizes  the  probability  of  playing  n 
games  before  going  broke,  for  all  n (stochastic  maximization).  The 
same  result  was  obtained  in  the  case  of  fair  games,  by  Freedman  [8] 
and  Molenaar  and  Van  Der  Velde  [13]. 
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Breiman  [2]  generalizes  Kelly's  model  and  proves  that  for  favour- 
able games,  the  gambling  system  which  maximizes  E(LnS  ) asymtoti- 

n 

cally  minimizes  the  expected  time  until  >_  x (for  sufficiently 

large  x)  and  asymptotically  maximizes  P[S  > y]  (for  sufficiently 

n 

large  n and  y fixed).  Breiman' s work  is  the  basis  for  a number 

of  papers  which  advocate  maximizing  ELnS^  in  the  belief  that  the 

gambler's  objectives  are  long  term. 

Ferguson  [7]  looks  at  the  optimal  strategies  for  coin  tossing 

games  in  which  the  gambler's  objective  is  to  avoxd  ruin.  He  assumes 

that  at  each  gamble  the  probability  of  winning  is  chosen  randomly 

from  a distribution  F(p)  and  told  to  the  gambler  before  his 

placing  a bet.  Ferguson  considers  two  cases:  the  gambler  must  pay 

$1.00  after  each  play  or  there  is  a minimum  bet  of  $1.00.  He 

conjectures  that  for  both  models  the  probability  of  ruin  tends  to 

zero  exponentially  as  the  gambler's  fortune  increases.  This  fact 

is  proved  by  Breiman  [3].  From  this  conjecture  Ferguson  proves  that 

if  F is  degenerate  at  some  point  p^  > h then,  for  his  first 

model,  if  the  gambler  is  wealthy  he  should  wager  approximately 
Ln((l  - P0)/pJ 

- — y~. ~ r-r-  . Truelove  [19]  extends  Ferguson's  paper  proving 

Ln(4pQ(l  - pQ)) 

that,  for  the  first  model,  the  probability  of  ruin  is  continuous 
in  x , the  gambler's  fortune. 

Thorp  [18]  gives  a good  overview  of  the  various  games  in  existence 
today  which  are  favourable  to  the  gambler. 


Epstein  [6]  has  written  a book  totally  devoted  to  the  various 
gambling  situations  available,  and  how  to  play  optimally  in  them. 


Pasternack  [14]  considers  the  coin  tossing  game  but  discounts 


money  over  time  and  finds  conditions  under  which  the  gambler  is  better 


off  by  making  risk-free  investments.  He  also  shows  that,  when  it  is 


optimal  to  gamble,  there  exists  a proportional  betting  strategy  such 


that  the  expected  discounted  fortune  converges  to  infinity  almost 


surely 


In  the  theory  of  consumption  and  investment  Hakansson  [9]  and 


Schechtman  [17]  have  considered  models  in  which  the  gambler  must  decide 


at  each  period  which  portion  of  his  fortune  to  consume  (and  derive  a 


utility  from)  and  which  portion  to  place  in  a risky  investment.  It  is 


interesting  to  note  that,  although  there  are  apparent  similarities 


between  their  models  and  those  of  a gambler,  their  results  differ  from 


those  for  the  gambler  whose  utility  is  paid  at  the  end  of  his  planning 


horizon.  This  can  be  explained  by  the  fact  that  such  gambler  lacks  an 


incentive  to  divert  part  of  his  funds  into  consumption 


This  thesis  is  divided  into  five  chapters  and  one  Appendix.  In 


Chapter  I we  formulate  the  gambling  model  using  the  dynamic  programming 


recursive  equation  method  and  find  some  general  properties  of  the  opti' 


mal  value  function 


Chapter  II  is  devoted  to  some  special  utility  functions  and 


generalizes  the  results  of  Bellman  and  Kalaba 


Chapter  III  considers  the  relation  between  the  gambler's  fortune 


Chapter  IV  examines  the  dependence  of  the  betting  strategy  on  the 


probability  of  winning  in  coin  tossing  games 
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Chapter  V generalizes  these  results  to  the  case  where  the 
probability  of  winning  is  unknown  except  in  distribution. 

Tha  Appendix  presents  results  due  to  Topkis  and  Veinott  which 
have  not  been  published  previously  but  are  fundamental  in  the  develop- 
ment of  this  work. 

Throughout  this  work  the  logarithms  considered  will  ba  natural 
logarithms  and  strategy  will  be  taken  to  mean  any  gambling  system 
whatsoever,  where  the  wagers  made  may  depend  on  the  history  of  the 
game,  current  fortune,  etc.  Following  Bourbaki,  we  will  use  the  terms 
"increasing,"  "decreasing,"  "greater  than,"  "less  than"  to  denote 
respectively  "nondecreasing,"  "nonincreasing,"  "greater  than  or 
equal  to,"  "less  than  or  equal  to,"  and  so  on,  reserving  the  word 
"strictly"  to  mean  exactly  that.  Thus  "increasing"  means  the  value 
does  not  decrease  (but  could  be  constant)  and  "strictly  increasing" 
means  the  value  does  not  decrease  and  is  not  constant  over  any 
interval . 

Lemmas,  theorems  and  corollaries  are  numbered  by  section  in  the 
order  in  which  they  appear.  The  Appendix  is  referred  to  as  Section  A. 
The  end  of  a proof  is  signaled  by  ■ , symbol  usually  attributed  to 
Halmos. 
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CHAPTER  I 

FORMULATION  OF  GAMBLING  MODEL 

Consider  the  following  scenario:  A gambler  has  a fortune  of 

x . He  knows  that  the  state  of  the  mechanism  against  which  he  is 
playing  is  i e {0,1,2,  ...}  . He  can  place  any  positive  wager  y 
not  exceeding  his  current  fortune  (0  y £ x)  . The  mechanism  will 
then  be  set  in  motion,  and  when  it  stop'*  his  fortune  will  be  x + Ry 
where  R is  a random  variable  with  distribution  G^(R)  , which 
depends  on  i . We  will  assume  P[R  ^ - 1]  = 1 , meaning  that  the 
gambler  cannot  loose  more  than  his  wager,  and  also,  that  /rdG^r)  < 

6 < “ , for  all  i . 

After  this,  the  state  of  the  gambling  mechanism  changes  to  j 
with  probability  P^  , independent  of  R . The  gambler  is  informed 
of  the  new  state  of  the  mechanism  and  is  allowed  to  place  another 
wager,  the  process  repeating  itself  a predetermined  number  of  times. 

The  gambler's  problem  is  thus  to  determine  his  betting  strategy, 
i.e.,  to  define  for  every  conceivable  situation  he  may  find  himself 
in,  the  amount  to  be  wagered. 

We  will  consider  a gambler  who  is  interested  in  maximizing  the 
expected  value  of  his  fortune  at  the  end  of  the  n plays,  or,  more 
generally,  we  will  assume  that  he  attaches  a utility  V(x)  to  each 
possible  value  of  his  final  fortune  and  he  wishes  to  maximize  his 
expected  utility  after  n gambles. 

The  utility  functions  we  will  consider  can  be  characterized  by 
the  properties 


a)  V(x)  increasing  in  x 

b)  V(x)  concave  in  x 

c)  V(x)  is  right  continuous  at  x ■ 0 . 
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Assumption  a)  simply  states  that  the  wealthier  the  gambler  is, 
the  happier  he  is.  Clearly  if  having  less  money  made  the  gambler 
happier,  he  could  adopt  the  strategy  of  giving  away  as  much  as  he 
needed  in  order  to  increase  his  utility.  Hence  we  have  no  use  for 
utilities  that  are  strictly  decreasing  over  any  subset  of  the  positive 
real  line. 

Assumption  b)  characterizes  utilities  with  decreasing  marginal 
returns.  It  states  that  if,  when  your  fortune  is  x , you  need  to  win 


Ax  in  order  to  increase  your  utility  by  e , then,  when  your  fortune 


is  x'  > x you  need  to  win  A , > A in  order  to  increase  your 

X X 


utility  by  the  same  amount.  In  simpler  terms,  if  you  were  poor  you 
would  be  inclined  to  accept  a menial  task  to  win  an  extra  $1,  but  if 
you  were  rich,  you  would  probably  not  accept  the  same  task  for  only 
one  extra  dollar. 

Assumption  c),  as  will  be  seen  below,  is  needed  in  order  to 
ensure  the  existence  of  optimal  betting  strategies.  It  does  not  of 
course  rule  out  the  possibility  of  unbounded  utilities  in  which 
V(0)  = - oo  . in  fact  we  will  take  - to  be  (by  continuity)  the 


B 


definition  of  LnO  and  of  - 0 g < 0 . 


Let  V^tx.i]  be  the  supremum  expected  utility  when  there  are  n 


betting  opportunities,  the  gambler’s  fortune  is  x and  the  mechanism 
is  in  state  i . The  usual  Dynamic  Programming  formulation  leads  us 
to  the  following  functional  equations 


Vx’i] 


" 

SUp  ^ pi1  f Vn-l{x  + ry*^dGj 

0<y<x  ( j ^ 


(r) 


V0[x,i]  - V[x]  . 


We  will  write 


^R  J / iVr 


'^[x  + Ry , J]  = l?±i  f V^tx  + ry,j]dG1(r) 


-1 


to  represent  the  expectation  with  respect  to  R and  J , whose 
distributions  depend  on  i . 

We  now  proceed  to  show  the  existence  of  optimal  policies,  as 
well  as  some  smoothness  properties  of  V^fx.i]  . 

Lemma  1.1; 

If  V^^tx.i]  is  continuous  in  x on  (0,°>)  and  right  contin- 
uous at  the  origin  for  all  i , then  there  exists  a number  y (x,i) 

n 

such  that 

"ll/iVl1*  + Ryn(Xll)’J]  = nSup  ERJ/iVltx  + Ry*J]  ’ 

0<y<x 

Proof : 

V^_^[x  + ry,j]  is  continuous  in  y e (0,°°)  and  right  continuous 
at  the  origin,  for  all  r and  j . Hence  so  is  E^j/^vn  + Ry»J]  • 

Since  {y  : 0 <_  y <_  x}  is  compact,  the  supremum  is  attained  . ■ 

Recall  that  J*rdG^(r)  <_  0 < » for  all  i . 


Define  t|/  = max  {1,1  + 8}  . Also  let  ER^  = J rdG^(r)  . 

Theorem  1.2: 

For  all  i , V^Jx.i]  is  increasing  and  concave  in  x . It  is 

thus  continuous  in  x on  (0,®)  . Furthermore,  V [x,i]  < V[x4>n] 

n — 

and  it  is  right  continuous  at  the  origin.  Hence  an  optimal  policy 
exists . 

Proof: 

Vq[x,1]  = V[x]  satisfies  all  the  above  by  definition.  Assume 

V [x,i]  does.  Then  the  number  y (x,i)  exists  for  all  x and  i 
n— 1 n 

by  Lemma  1.1. 

Now,  for  e > 0 , since  0 £ y^Cx.i)  £ x + e 

Vn[x  + e,i]  > E^j/^Ix  + c + Ryn(x,i),J] 

i ERJ/iVn-l[x  + Ryn(x’ 1)  ,,Tl  = Vntx’i] 

hence  V [x,i]  is  increasing  in  x . 
n 

For  any  x^  , x2  and  any  X e [0,1]  , since  0 <_yn(x^,i)  <_  x^ 
and  0 <_  yn(x2,i)  <_  x2  imply 

0 <_  Ayn(xlfi)  + (1  - X)yn(x2,i)  £ Ax^^  + (1  - X)x2 

showing  that  Xy  (x_,i)  + (1  - A)y  (x-,i)  is  a feasible  wager  when 
n J.  n ^ 

our  fortune  is  Ax.  + (1  - X)x-  , we  have 


■ v; 
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Vn^Xl  + ^ ” XjXj.i)  L 

ERJ/iVn-l^Xxl  + ^ “ X^x2  + R(Xyn(xl’i)  + ^ " *)yn(x2,i)),J] 
and  by  concavity  of  vn  ^[X«J]  in  the  first  argument 

— ERJ/i  ^XVn-l^Xl  + Ryn(xl,i),J]  + ^ X^Vn-l^X2  + Ryn^X2’i^,J^ 
showing 


Vn[Xxl  + (1  ’ X)x2»i)  > Wjx^l]  + (1  - X)V.lx2, i]  . 

Thus  V^tx.i]  is  concave  in  x , hence  continuous  in  x , except 
possibly  at  the  origin. 

Since  V [x,i]  is  increasing  in  x , clearly  lim  V Ix,i] 
n x->-0+  n 

exists  (it  may  be  - «>)  . It  is  also  evident  that  V [0,i]  = V[0]  . 

n 

By  the  concavity  of  we  have  as  a consequence  of  Jensen' 

inequality  ([12],  p.  159)  that  for  all  x > 0 and  y e [0,x]  , 


CO 

? p«  / V iIx  + i I pijVi[lt  + “i*- 

J -1  j 


j] 


P«Vn-lIx  + 


For  ye  [0,x]  we  have  thus, 


oo 

I pi:i  f vn_i[x  + ry*^dGi<r)  il  pijvn_i[x,J'» j]  iv[xfni 
j -i  j 


showing 
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V [x,i]  <_  V [x«^n]  for  all  x . 


Now,  if  V(0)  - V [0,1]  > - • , then 
n 


0 < V [x.i]  - V [ 0 , i]  < V[xiT]  - V[0] 
— n n — 


and  letting  x go  to  zero. 


lim  V [x.i]  = V [0,i] 
x+0*-  ” n 


If  V[0]  * V [0,i]  * - <«>  then 
n 


7ntx,i]  <_  V[x^  ] 


and  letting  x go  to  zero 


lim  V [x.i] 
x^0+  n 


- oo  * v [0,i] 

n 


which  proves  the  right  continuity  at  the  origin . 


It  thus  follows  that  the  supremum  in  the  optimality  equation  is 
always  attained  and  we  can  and  will  write  it  as 


v (X.ij  - MX  E av  U + Ry,Jl 
0<y<x 

Vq[x,1]  = V[x]  . 


We  will  call  y^Cx.i)  the  optimal  amount  to  bet  as  determined 
from  this  equation.  If  it  is  not  unique,  we  will  allow  y^(x,i)  to 
be  either  the  smallest  or  the  largest  of  such  values,  although  we 
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will  assume  consistency  in  this  choice:  always  the  smallest  or 

always  the  largest. 


Proposition  1.3: 


V^lx.i]  is  increasing  in  n 


Proof : 


By  deciding  to  bet  zero  on  the  last  stage  of  an  n stage 
problem,  one  reduces  it  to  an  (n  - 1)  stage  problem.  Thus  in  the 
n stage  problem  one  can  do  at  least  as  well  as  in  an  (n  - 1) 
stage  problem.  ■ 


Proposition  1.4: 


Whenever  the  system  is  in  a state  i such  that  ER^  <_  0 then 
it  is  optimal  to  bet  zero. 


Proof : 


Vn[x,i]  * max  l P^ER/iV^tx  + Ry,j] 
0<y<x  j 


Since,  by  Theorem  1.2  V .[*,*]  is  concave  in  the  first  argument, 

n- 1 

we  may  apply  Jensen's  inequality  (cf.  [12],  p.  159)  to  obtain 


/ [x,i]  < max  £ P..V  [x  + (ER  )y,j] 
n — . , 11  n-l  l 

0<y<x  j J 


Since,  by  Theorem  1.2  V ,[x,j]  is  increasing  in  x for  all  j 

n-i 

and  ER^  0 we  have  that 
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^[x  + (Er^)y,j]  is  decreasing  in  y for  all  j hence 
£ PijVn  + (ER^yJ]  is  decreasing  in  y and  its  maximum  over 
0 < y < x is  attained  at  y = 0 . Thus 

VX,i]  - E PijVn-ltx,j]  ’ 

However,  since  betting  zero  is  always  a feasible  strategy  we  must 
have 

Thus  we  must  have  that  ER^  <_  0 implies 

V*'11  ’ £ PijVltx’JI 

showing  the  optimality  of  betting  zero . O 

This  last  result  is  of  course  the  reasor  for  which  concave 

utility  functions  are  sometimes  called  "risk  averse"  (see  for  example 

[15]),  A gambler  whose  utility  has  this  property  will  not  bet  on  a 

gamble  where  his  expectation  is  negative.  From  now  on,  unless 

specifically  indicated,  we  will  assume  ER^  > 0 , and  consider  this 

condition  as  the  definition  of  a favourable  gamble. 

So  far  we  have  shown  that  V [x,i]  inherits  the  monotonicity 

n 

and  concavity  properties  of  Vg[x,i]  . Another  property  which  will 
prove  useful  later  is  differentiability. 


To  this  end  we  establish  first 


I 
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Lemma  1.5: 

If  Vn_^[x,i]  strictly  increasing  and  differentiable  in  the 
first  argument  at  some  > 0 for  all  1 , then  y^Xg.i)  > 0 
for  all  i . 

Proof: 

VX0*i]  = ““  ERJ/iVl[x0  + Ry’J]  • 

°iyix0 

Since  the  function  to  be  maximized  is  differentiable,  we  can  compute 
its  derivative  with  respect  to  y , evaluated  at  y = 0 to  be 

00 

-1  J 

- <ev  \ vi-i'v11  * 0 

since  ER^  > 0 and  ^[x^.j]  > 0 for  all  j . Hence 

i tXQ  + is  strictly  increasing  in  y at  y = 0 and 

the  optimal  y must  be  strictly  positive.® 

We  can  now  prove 

Theorem  1.6: 

If  V[x]  is  strictly  increasing  and  differentiable  at  x > 0 
then  so  is  V^tx.i]  for  all  n and  all  i . 

Proof : 

Our  proof  follows  that  of  Schechtman's  theorem  2.2  [17].  Vq[x,1]  = 

V[x]  is  strictly  increasing  and  differentiable  at  x > 0 for  all  i . 


r 


s r. 
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Assume  V^^fx.i]  has  this  property  at  Xg  . By  Theorem  1.2 
Vjx,i]  is  concave.  Thus  it  will  be  differentiable  at  Xg  if  and 
only  if  it  has  a unique  support  at  this  point. 

Let  y = yn(xg,i)  , u = Xg  - y . By  Lemma  1.5  y > 0 . 

Let  Sg  be  a support  of  V^tx.i]  at  Xg  . By  concavity  of 
V^tx.i]  we  must  have  that,  for  any  x >_  0 

VntX,i]  - SgX  lVn[Xg,i]  - SgXg  . 

For  any  y >_  0 , u >_  0 , setting  x = y + u we  also  must  have, 
since  y is  not  necessarily  the  optimal  bet  at  x 


Wn-i1' + Ry,J1  iV-'11  • 

Substituting  above  we  obtain 

ERJ/iVl[u+(1  + R)y’Jj  ‘ S0U  ’ V -ERJ/iVn-ltG+(1  + R)y’J]~  V ~ V 

Since  u was  arbitrary,  we  can  set  it  equal  to  u obtaining 
that  for  all  y >_  0 

ERJ/iVn-lt“  + (1  + * V i + (1  + R)y-”  - V • 

Thus  we  see  that  the  differentiable  concave  function  of  y , 
RRJ/i^n  1^U  + ^ + R)y»J]  “ SQy  attains  its  maximum  over  y >_  0 at 
y > 0 . This  implies  that  its  derivative  with  respect  to  y vanishes 


■W1  + + a + R)y'Jl  - so  ■ 


at  y , hence 


0 
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or 

so  ■ Enj/i<i  + 1*0  + R^J1  ■ 

Thus  the  support  Sq  must  be  unique,  showing  the  differentiability 

of  V [x,i]  • We  also  obtain  a useful  recursive  equation  for  the 
n 

derivatives  of  V^fx.i]  , namely, 

V’Jx.i]  = Epj/i(l  + R)V^_x[x  + Ry,J]  . 

Also,  since  ^[x  + Ry,J]  > 0 and  (1  + R)  >_  0 , we  see  that 

V'[x,i]  > 0 (unless  P[R  = - 1]  = 1 which  is  ruled  out  by  the  fact 
n 

that  ER.  > 0)  showing  that  V [x,i]  is  strictly  increasing  in  x . ■ 
i n 

We  now  turn  to  the  examination  of  the  dependence  of  V [x,i] 

n 

on  its  second  argument.  Clearly  not  much  can  be  said  unless  one 
imposes  some  structure  on  the  random  variables  that  determine  the 
outcome  of  the  gamble.  The  type  of  result  we  are  interested  in  is, 
in  broad  terms,  "the  better  the  gamble  available  to  you,  the  better 
off  you  are."  To  this  effect  we  will  assume  for  the  remainder  of 
this  section  at  the  random  variables  R^  are  stochastically  increasing 
in  i , or,  in  mathematical  terms  PtR^  ^ r]  is  increasing  in  i 
for  all  r . 

As  is  well  known,  if  $ is  an  increasing  function  then  the 
stochastic  ordering  of  the  R^  implies  fc4>(R^)  is  increasing  in  i . 

The  application  of  stochastic  monotonicity  to  markov  chains 
yields  the  concert  of  monotone  matrices  (Keilson  and  Kester  [10], 

Daley  [4]).  The  transition  matrix  of  a markov  chain,  P = [ P ^ ^ ] 


is  said  to  be  monotone  if  and  only  if  £ P is  increasing  in  i 

j=k  iJ 

for  all  k . This  merely  says  that  the  rows  of  the  matrix  correspond 
to  the  distributions  of  a stochastically  increasing  sequence  of 
random  variables  or,  in  simpler  terms,  the  higher  the  state  of  the 
system,  the  stochastically  higher  the  state  it  will  jump  to  next. 

It  thus  follows  that  if  f is  an  increasing  sequence  and  if  P 
is  monotone,  then  J f.P..  is  increasing  in  i . 

j 3 3 

We  are  now  in  a position  to  prove 


Proposition  1.7: 

Assume  the  are  stochastically  increasing  in  i and  the  matrix 

of  transition  probabilities  is  monotone.  Then  V [x,i]  is  increasing 

n 

in  i . 


Proof : 

Vg[x,i]  = V[x]  is  increasing  in  i . 

Assume  V ^[x,i]  is  increasing  in  i . 

Then  it  follows  that 

/ V^tx  + ry,j]dG.(r) 

j . If  y ^ 0 then,  by  Theorem  1.2  ^Lx  + ry,j] 
r . Therefore,  by  the  stochastic  monotonicity  of 

/vn_xtx  + ry,j]dGi(r) 


is  increasing  In 
is  increasing  in 


is  increasing  in  i , as  well  as  in  j . 
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By  the  monotone  property  of  the  transition  matrix 


l /vn-i[x  + ry«jJdGi(r> 


j 


v lx.i]  * max  £ p ./v  .[x  + ry,j]dG  (r) 

0<y<_x  1 


is  increasing  in  i for  all  y ^ 0 . Thus 

ij 

v - jr  -a 

is  increasing  in  i . ■ 

As  a special  case  of  the  above  proposition  we  see  that  if  the 
state  of  the  gambling  system  does  not  change  form  play  to  play,  then 
for  stochastically  greater  gambles  we  obtain  greater  expected 
utilities.  This  follows  from  the  fact  that  the  transition  matrix 
in  this  case  is  the  identity  matrix  which  is  monotone. 

It  is  a trivial  task  to  construct  counter-examples  to  show  that 
the  result  no  longer  holds  if  one  merely  assumes  that  the  gambles 
are  ordered  stochastically  and  does  not  impose  special  conditions 
on  the  transition  matrix. 

Let  the  utility  function  be  V[x]  = x . As  a consequence  of 
Theorem  2.2  we  will  see  that  in  this  case  one  should  always  bet  his 
current  fortune  (if  the  gamble  is  favourable)  i.e.  y^(x,i)  ■ x Vn  , 
Vi  3 ERi  > 0 . 

The  three  gambles  available  are  coin  tosses  with  probabilities 
of  winning  .6,  .7  and  .8  respectively.  Thus,  clearly,  the  correspond- 
ing random  variables  are  stochastically  ordered. 

The  transition  matrix  will  be  taken  as 
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P 


10  0 
0 10 
.1  o 0. 


which  is  clearly  not  monotone. 

It  is  easy  to  verify  then  that 

Vn[x,2]  = (.7)n2nx 

V [x,3]  = .8(.6)n_12nx 
n 

and  obviously  V [x,3]  < V [x,2]  for  all  n > 2 which  is  true 
n n — 

because  after  playing  the  best  bet  once,  one  is  forced  to  continue 
with  the  worst  one  from  then  on,  whereas,  starting  with  the  second 
best  bet  one  will  always  face  that  one. 


CHAPTER  II 


SOME  SPECIAL  UTILITY  FUNCTIONS 


As  a preamble  to  the  analysis  of  the  betting  strategy  in  the 
general  case,  we  consider  some  special  forms  of  utility  functions 
which  seem  to  be  fairly  popular  among  economists.  We  first  consider 
V[x]  = Lnx  which  was  essentially  the  utility  analyzed  by  Kelly. 


Theorem  2.1: 


If  V[x]  = Lnx  then  V [x,i]  * V [l,i]  + Lnx  and  the  optimal 

n n 

wager  is  given  by  y^Cx.i)  = a(i)x  where  a(i)  does  not  depend  on 
n or  x . 


Proof : 


Instead  of  choosing,  at  each  stage,  the  amount  to  bet  the  gambler 
could  equivalently  take  as  his  decision  variable  the  fraction  of  his 
current  fortune  to  be  wagered.  The  dynamic  programming  recursive 
equations  now  become 


V [x , i]  = max  E , V [x(l  +aR),J]  . 
n 0<a<l  n'1 


The  proof  then  proceeds  by  induction. 

Vg[x,i]  = Lnx  which  is  clearly  of  the  required  form  since 

VQ[l,i]  = Lnl  = 0 . 

Assume  V . [x,i]  ■ V [l,i]  + Lnx  . 
n-i  n-I 

VX’1]  = ““  {ERJ/iVn-l[1’J]  + ERJ/iLntx(1  + aR)1} 

ae[0,lj 

= ERJ/iVn-l[1,J]  + Lnx  + ERJ/iLn(1  + aR) 

ae[0,l] 


f 


21 


By  setting  x * 1 we  have,  since  Lnl  » 0 , that 


V1*11  ' Wn-l11"11  + W*1  * «*> 


which,  compared  to  the  equation  preceeding  it  shows  that 


V [x,i]  =•  V [1, i]  + Lnx 
n n 


thus  completing  the  induction.  Note  also  that  the  value  of  the 
optimal  a is  determined  by 


max  E^^LnCl  + aR) 
ae[0,l] 


which  shows  that  the  fraction  of  the  current  fortune  to  be  wagered 
in  independent  of  n and  x . ■ 


This  result  generalizes  that  of  Bellman  and  Kalaba  [1]  for  coin 
tossing  games.  It  is  interesting  to  note  that  the  policy  is  absolutely 
micpic,  that  is,  it  does  not  consider  the  planning  horizon  or  the  possi- 
bility of  the  appearance  of  better  (or  worse)  gambles  in  the  future. 

O 

Another  interesting  utility  is  x°/B  for  a constant  B <_  1 , 
6^0.  It  is  obvious  that  the  utility  considered  above  is  a limiting 
case  of  this  one,  as  B goes  to  0 . A similar  result  to  the  one 
above  holds  for  this  utility. 


Theorem  2.2: 


If  V[x]  = xB/B  B < 1 B 4 0 then  V [x,i]  = V [l,i]xB  and  the 
— n n 

optimal  wager  is  given  by  yn(x,i)  = a(i)x  where  cc(i)  does  not 
depend  on  n or  x . 


***"*— Tom  ■» 


I 
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Proof : 


We  will  only  consider  the  case  6 > 0 . The  proof  in  the  case 
6 < 0 is  identical  except  that  one  must  consider  the  fact  that  the 
utility  is  negative. 

Formulating  the  problem  in  terms  of  the  fraction  of  the  current 
fortune  to  be  wagered  yields  as  before 


V*'11 ' Sj/iVi1'11  + 

ae[0,A] 


16  R 

V^[x,i]  i — x is  of  the  required  form  since  1P  >*  1 


Assume  Vn_^[x,i]  = V ^[l,i]x 


6 


V [x,i]  = max 
n qe[0 


K I f Vn  1 [1»j]  t*(l  + 

,l](j  ij^  n_1 


ar) 3 PdG±(r) ] 


(l  p^vn  [l,j])  max  f (1  + ar)edG  (r) 
V 13  n_1  ’ ae [0,1] 


By  taking  x = 1 one  sees  that 


V1.!] 


(l  PijVn-lU’JI)  /(1  + “r>6'iGi<r) 

ae[0,l] 


and  clearly  V [x,i]  = V [l,i]xP  . 
n n 

Also  the  optimal  fraction  to  be  bet  is  determined  by 


max  / (1  + ar)PdG  (r) 

actO.l] 


,6 
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which  does  not  depend  on  x or  n implying  a(i)  does  not  depend 
on  these  parameters. ■ 

In  the  case  6 < 0 , by  considering  the  negativity  of  V [l,i] 

n-i 

in  the  same  way  as  in  Theorem  2.3  below  we  conclude  that  the  optimal 
a is  determined  by 


min  / v.1  + ar)^dG  (r)  . 

at [0,1] 

As  a consequence  of  Theorem  2.2  we  note  that  if  V[x]  = x 
(i.e.  6=1)  then  one  should  always  bet  his  current  fortune,  for 

then  a(i)  is  determined  by 


max 

a 


/ 


(1  + ar)dG^(r) 


max  (1  + aER^) 
a 


if  ER.  > 0 then  the  maximum  is  attained  at  a = 1 . It  is  interest- 

l 

ing  to  note  that  it  is  precisely  this  strategy  that  maximizes  the 
probability  of  ruin  over  any  horizon! 

Another  utility  function  sometimes  found  in  the  literature  is 
V[x]  = 1 - e , or,  since  adding  a constant  to  the  utility  function 

“X 

does  not  alter  the  optimal  strategy,  its  equivalent  - e . This 
utility  does  not  yield  simple  strategies,  unless  the  gambler  is 
assumed  to  have  infinite  credit,  that  is,  he  is  allowed  to  bet  beyond 


his  current  fortune. 
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Theorem  2.3: 

Assume  that  V[x]  * - e and  that  the  gambler  is  allowed  to 
wager  any  positive  amount.  Then  V^[x,i]  » V^tO.ije  x and  the 
optimal  wager  y^Cx.i)  is  independent  of  n and  x . 

Proof : 

Vg[x,i]  * - e is  clearly  of  the  required  form.  Assume 

V ,[x,i]  = V ,[0,i]e  X . Since  the  utility  is  negative,  we  must 
n-l  n-i 

have  ^[0,i]  0 . Now 


V*-11 


00 


max  l P f V [0,j]e“(x+ry)dG  (r) 
y>0  j 2 J 1 1 


Vn[x,i] 


-X 


max 

yi°  (j 


e"rydG1(r>; 


-1 


Since  V [0,j]  <_  0 it  follows  that 


l i 0 


Therefore 
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By  taking  x » 0 we  hrve 


V0-11  - (I  Vn-lI0>J1)  f ''ryjGl<r) 


and  thus 


V [x,i]  = V [0,i]e'X  . 
n n 


We  also  note  that  the  optimizing  y , say  y(i)  is  chosen  from 


min  / e ^dG  (r) 

yi°  _i 

and  does  not  depend  on  x or  n . From  Lemma  1.5  one  concludes 
that  this  maximizing  value  is  strictly  positive.® 

As  the  following  corollary  points  out,  if  the  gambler  is  suffi- 
ciently wealthy,  then,  under  this  utility,  he  should  always  bet  a 
fixed  amount  which  depends  only  on  the  currently  available  gamble, 
even  if  no  credit  is  available. 

Covollary  2.4: 

Assume  that  V[x]  = - e X and  that  no  credit  is  allowed.  Let 

•k 

y(i)  be  as  in  Theorem  2.3  and  let  y = sup  y(i)  assumed  finite. 

* 

Then  x >_  (n  - l)y  + y^  implies  y^Cx.i)  = y(i)  . 


■ 


f I 


i 

I 


[ 


[ 
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Proof : 

* 

The  proof  hinges  on  the  fact  that  x >_  (n  - l)y  + y(i)  implies 
that  betting  y(j)  whenever  in  state  j will  be  feasible  whatever 
the  outcome  of  the  n games  in  the  planning  horizon.  Since  this 
strategy  is  optimal  in  the  (less  restricted;  situation  where  credit 
is  available,  its  optimality  follows.  ■ 

We  have  thus  seen  that  the  optimal  strategy  for  all  these  utility 
functions  can  be  considered  to  depend  essentially  on  the  distribution 
of  the  gamble  currently  available.  An  interesting  question  that  arises 
is  how  do  these  strategies  depend  on  this  gamble,  that  is,  if  I have 
a better  gamble  available,  should  I bet  more?  This  intuitively 
appealing  question  is  answered  affirmatively  (under  some  conditions) 
in  the  following  theorems. 

Theorem  2.5; 

Assume  the  random  variables  are  stochastically  increasing  in 

i . Assume  furthermore  that  V[x]  » Lnx  . Then  the  optimal  proportion 
of  the  current  fortune  to  be  wagered,  a(i)  is  increasing  in  i . 

Proof : 

From  the  proof  of  Theorem  2.1  it  follows  that  a(i)  is  determined 
by 

max  ELn(l  + aR  ) . 
ae[0,l] 


l 


. 


] 


s 


■ 


r 


Since  the  set  {(a,i)  | a e [0,1]  , i = 1,2,3,  ...}  is  clearly  a 
sublattice,  the  result  will  follow  from  the  isotonicity  theorem 
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(A. 11)  if  ELn(l  + aR^)  is  superadditive  on  this  sublattice,  and, 
from  Theorem  A. 3,  this  will  be  true  if,  for  t > 0 such  that 
(a  + e)  e [0,1]  , 


ELn(l  + (a  + e)R^)  - ELn(l  + aR^)  = ELn 


1 + (a  + e)Ri 
1 + aR, 


is  increasing  in  i for  a and  e fixed. 

Consider  the  function  (1  + (a  + e)r)/(l  + ar)  which  is  clearly 

continuous  and  differentiable  in  r . Its  derivative  with  respect  co 

r is  given  by  j >0  so  the  function  is  increasing  in  r . 

(1  + ar) 

„ /l+(a  + e)r 

Thus  Ln  [ i-r.," 

/I  + (a  + e)Ri\ 

are  stochastically  increasing  in  i ELn  ( ^ + aR ) is  increasin6 


^ is  also  increasing  in  r and  since  the  R^ 


in  i . ■ 


The  above  result  can  be  considered  a generalization  of  Bellman 
and  Kalaba's  result  for  coin  tossing  games:  in  this  case  one  should 

bet  the  fraction  2p  - 1 where  p is  the  probability  of  winning. 
Since  Bernourilli  random  variables  are  stochastically  increasing  in 
p it  is  not  surprising  that  their  optimal  fraction  is  also  increasing 
in  p . 

Theorem  2,6: 

Assume  the  random  variables  R^  are  stochastically  increasing  in 

C 

i . Furthermore  assume  that  V[x]  =x/B  8 < 1 8^0.  Then, 


i 
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calling  a(i)  the  optimal  fraction  of  the  current  fortune  to  be 
wagered 


a)  if  6 > 0 then  a(i)  is  increasing  in  i 


b)  if  6 < 0 and  Pj^ 
is  increasing  in  i . 


Ri  > - ? 


0 for  all  i , then  a(i) 


Proof : 

In  the  case  8 > 0 , a(i)  is  determined  by 


max  E(1  + aR  ) 
ae[0,l] 


and,  as  in  Theorem  2.5,  will  be  increasing  if  E(1  + (a  + e)R^)P 

O 

E(1  + aR^;  is  increasing  in  i which  will  in  turn  follow  if 
8 8 

(1  + (a  + e)r)p  - (1  + ar)  is  increasing  in  r . Clearly  this 
is  equivalent  to  showing  that 


(1  + ar)®  = 6(1  + ar)g  1r 


3 6-1 

is  increasing  in  r . To  show  this,  we  compute  — 8(1  + ar)  r 

<7  r 


and  show  it  is  nonnegative. 


6(1  + ar)®  ^r  = 6 ( B - 1)  (1  + ar)®  ^ar  + 8(1  + ar)®  * >_  0 


if  and  only  if 


(8  - l)ar  + 1 + ar  >_  0 


or  equivalently 


F 


mm 
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Bar  + 1 0 

which  follows  from  the  fact  that  0 < 3 <_  1 , 0 < a < 1 , R >_  - 1 . 
In  the  case  B < 0 , a(i)  is  determined  by 

min  E(1  + aR  )6  . 
ae[0,l] 

By  the  remarks  at  the  end  of  the  appendix  a(i)  will  be  increasing 
in  i if  E(1  + aR^)P  is  subadditive  over  the  sublattice  of  pairs 
(a,i)  , or,  in  other  words,  if,  for  e > 0 

E(1  + (a  + e)Ri)B  - E(1  + aR±)B 

is  decreasing  in  i . This  in  turn  will  be  true  if  the  function 

(1  + (a  + e)r)^  - (1  + ar)^ 

is  decreasing  in  r , because  of  the  stochastic  monotonicity  of  R^  . 
Equivalently  it  suffices  to  show  that 

(1  + ar)3  = Br(l  + ar)6"1 

is  decreasing  in  r , that  is,  if  its  derivative  with  respect  to  r 
is  negative 

6(1  + ar)^  1 + 8(6  - l)ar(l  + ar)^  ^ <_  0 


or.  equivalently,  since  B < 0 
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1 + ar  + (6  - l)ar  > 0 


or 


r < 


which  is  true  since  by  assumption 


R 


< - 
i - 


1 

a 


because  a <_  1 . ■ 

The  intuitive  reason  behind  the  need  for  this  upper  bound  on 
the  outcome  of  the  gamble  in  the  case  6 < 0 can  best  be  explained 
by  considering  an  example  in  which  we  assume  the  bound  is  violated. 
Take  6 * - 1 and  consider  only  gambles  of  the  form: 

{r  with  probability  P 
- 1 with  probability  q * 1 - p . 

Assuming  rp  - q > 0 (i.e.  ER  > 0),  the  optimal  a is  determined 

by 

min  | t— T + ^ | 

aclO.l]  I 1 + “r  1-“) 


and  is  easily  found  to  be  given  by 


. _ jr  - (1  + rjv'pqr 

a = m *.n  < r1— k— 

| r(p  - qr) 


. 1 • 


Now  take  p = .9  q 


.1  . 
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k 


Gamble  1 

has 

r - 2 

and 

a - .5194  . 

Gamble  2 

has 

r = 3 

and 

a = .5119  . 

Clearly  R2  is  stochastically  larger  than  and  yet  it  is 

optimal  to  bet  less.  The  reason  now  is  apparent:  since  the  utility 

is  bounded  above  (by  0) , then  betting  more  in  gamble  2 than  in 
gamble  1,  if  we  win,  does  not  increase  our  utility  by  much  more, 
and  if  we  loose,  the  decrease  in  utility  is  larger.  Therefore  we 
sacrifice  some  of  the  possible  winnings  in  order  to  hedge  against 
possible  losses,  showing  once  more  a conservative  behavior. 

It  is  interesting  to  note  that,  in  spite  of  the  above,  for 
coin  tossing  games,  Bellman  and  Kalaba  [1]  find  that 


a(i) 


1 1 
1"6  n1_B 

p,  - q. 


1 

,1-6 


which  is  seen  to  be  increasing  in  i if  p^  is  increasing  in  i . 

“X 

For  the  utility  function  V[x]  = - e , it  is  easy  to  see  that 
the  amount  y(i)  to  be  wagered  is  not  increasing  in  i , even  if 
the  R^  are  stochastically  increasing.  In  fact  y(i)  is  determined 
by 

"V 

min  Ee 
y>0 

Assuming  R^  has  the  same  distribution  as  above,  y(i)  can 
be  computed  by 


i. 


t 


1 

| 


1 
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min  {pe  + qe^} 
y>0 


and  is  easily  seen  to  be  given  by 


y * max  { (Lnr  + Lnp  - Lnq)/(1  + r),0} 


For  the  same  gambles  as  before  we  obtain 


y (1)  = .9634 
y(2)  - .8240  . 


Hence  it  is  evident  that  some  conditions  are  needed  to  insure 
monotonicity. 


Theorem  2.7: 


Assume  the  random  variables  are  stochast? cally  increasing 

in  i . Assume  furthermore  that  V[x]  * - e 

Suppose  that  the  gambler  has  unlimited  credit,  or  that  his  for- 
tune is  large,  as  per  Corollary  2.4.  Let  y(i)  be  the  optimal  amount 


to  wager  and  y * sup  y(i)  assumed  finite.  Then  if  P 
y(i)  is  increasing  in  i . 


" 0 
y . 


Proof : 


-\y  * 

y(i)  is  determined  by  min  Ee  . Since  y(i)  <_  y we  can 

y>0 

clearly  reduce  this  problem  to 


min  ^ Ee 


0<y<y 


is  subadditive  in 


This  will  follow  by  the  stochastic  monotonicity 


or  equivalently  if 


which  will  follow  if 


This  will  be  true  if  r < 


over  the  sublattice  under  con- 


sideration. But  on  this  sublattice  y <_  y implying 


which  in  turn  implies  r <_ 


Note  that  this  bound  depends  on  the  distributions  of  R 


and  its  computation  requires  the  solution  of  the  problem,  which  will 


answer  directly  the  questions  of  monotonicity  of  y(i)  . Thus  the 


value  of  this  result  is  only  of  theoretical  interest 


For  coin  tossing  games,  one  easily  finds  y(i)  = Lnp.  - Lnq 


which  is  clearly  increasing  in  i if  p is 


CHAPTER  III 


DEPENDENCE  OF  THE  OPTIMAL  STRATEGY  ON  THE  FORTUNE 


We  now  return  to  the  general  problem  of  Chapter  I and  d Iscuss 


the  dependence  of  the  optimal  amount  to  bet,  y (x,i)  on  the 


gambler's  fortune  x 


From  the  results  of  the  previous  chapter,  it  is  tempting  to 


Both  of  these  are  false,  as  shown  by  the  following  example 


Let  the  gamble  be  a coin  tossing  game  with  probability  .6  of  winning 


It  is  an  easy  matter  to  compute  the  optimal  strategy  for  one 


stage  horizon  as 


which  is  clearly  not  monotone  in  x 


The  optimal  fraction  to  bet  is 


W0? 


, 


1 


a(x)  - - y^x) 


.2  - - 1 

x 

1 - — 

X 


0 < x <_  1/6 
1/6  < x < 3/4 


3/4  < x 


which  is  clearly  not  monotone  in  x . 

However,  under  some  conditions,  we  can  find  functions  that 
depend  on  the  optimal  policy  and  exhibit  simple  monotone  behaviour. 

Let  Rj.  be  a random  variable  P[R^  >_  - 1]  =1  . Define 
. p.  = sup  {r  : P[0  < R^  <_  r]  = 0}  . In  other  words  R^  has 
probability  0 of  being  strictly  positive  and  srraller  than  pi  and 
p^  is  the  largest  of  all  such  values. 

Theorem  3.1: 

Assume  the  state  of  gambling  system  is  i . Assume  p^  > 0 . 

Then,  if  y (x,i)  is  the  optimal  amount  to  bet  x + p y (x,i)  is 
n in 

increasing  in  x . 


i 


Proof : 


V [x,i]  = max  E . V [x  + Ry,J]  . 
n 0<y<x  R,J/i 


For  y >.  0 , let  v = x + p y . Then  y e [0,x]  implies 


v e [x,x(l  + p ] and  we  have 


JnU,l]  - -ax  ^ ''.J] 


subject  to  x <_  v <_  x(l  + p.)  . 


Note  this  constraint  set  is  a sublattice  of  pairs  (x,v)  by 


tramples  A1.4  and  Lemma  A.l. 


Now 

consider 

v4(‘  • 

■ fiY  4 v,j]  • F°r  r - 

p / 

0 

_ V 

> 0 and 

— < 0 . 
pi  - 

Thus  in  this  case  V , x|] 

n_1L  \ 

l'^r)+ 

f;  H 

is  superadditive  in 

x and  v on  our  sublattice,  because 

Vi[x>jl 

| is  concave  in  x 

and  Example  A2.5  applies. 

For 

R > 0 , 

we  need  only  consider  R >_  p^  by  the 

definition 

of  p±  . 

In  this 

R „ 
case  — > 

pi  " 

1 and  thus 

again  superadditive  in  v and  x by  the  same  reasons.  Since 
mixtures  of  superadditive  functions  are  clearly  superadditive,  we 
have  that 


E 


RJ/iVn-l 


is  superadditive  in  x and  v on  the  sublattice 

{(x,v)  | x£v£x(l  + pi)}  . 

Thus,  by  the  isotonicity  theorem  (A. 11)  v^Cx.i)  , the  optimizing 
value  is  increasing  in  x . This  optimal  value  is  clearly 
v (x,i)  - x + p y (x,i)  .■ 
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Note  that  if  the  optimizing  value  of  y is  not  unique  (in  which 
case  there  must  be  an  interval  of  maximizing  values  because  of  the 
concavity  of  the  function  being  maximized)  then  the  isotonicity  theorem 
guarantees  that  we  may  choose  the  largest  of  such  values,  or  the 
smallest  and,  as  long  as  we  are  consistent  in  this  choice,  the  result 
will  still  hold. 

The  function  given  is  tight,  in  the  sense  that,  if  p^  > 0 and 

p > p then  there  exist  problems  for  which  x + py  (x,i)  is  not 
i n 

increasing  in  x , as  the  following  examples  shows. 

Let 

!2x  x 4 

6 + H x 4 < x . 

Assume  only  one  stage  and  let  the  gamble  available  be  given  by 

1 with  probability  h 

- h with  probability  h . 

It  is  easy  to  see  that  the  optimal  betting  strategy  is 

0 1 x 1 2 
2 < x <_  4 

4 < x 8 
8 < x . 

Clearly  pi  = 1 . For  p > 1 , in  the  region  2 < x <_  4 

x + py^(x,i)  * x(l  - p)  + 4p  decreasing  in  x . 

Hence  our  result  is  tight  in  the  sense  described  above. 


f 
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The  decision  variable  we  have  considered  so  far  is  the  amount 

to  bet  y (x,i)  . An  alternative  would  be  to  decide  how  much  not 
n 

to  bet,  and  wager  the  remainder. 

Let  u (x,i)  = x - y (x,i)  . Examining  the  results  of  the 
n n 

previous  chapter,  one  is  tempted  to  conclude  that  this  quantity 
is  an  increasing  function  of  x . This  is  not  generally  true,  as 
can  be  seen  from  the  last  example.  In  the  region  4 < x 8 
u^(x,i)  * 8 - x , decreasing  in  x . 

However,  in  the  same  spirit  of  Theorem  3.1  we  have  the  following 
result. 

Let  R^  be  a random  variable  P[R^  >_  — 1]  = 1 . Define 
Ci  « | inf  {r  : P[r  £ R±  < 0]  « 0}|  . 

In  other  words  R^^  has  probability  0 of  being  strictly 
negative  and  larger  than  - and  is  the  largest  of  all  such 

values.  0 <_  ^ <_  1 by  definition,  if  it  exists. 

Theorem  3.2; 

Assume  the  state  of  the  gambling  system  is  i . 

Assume  £.  exists  and  £.  > 0 . Then  if  y (x,i)  is  the 
li  n 

optimal  amount  to  bet,  x - £.y  (x,i)  is  increasing  in  x . 

l n 

Proof ; 

From  the  optimality  equation 


Vj/iVi1*  + Ry’J1  ' 


By  making  the  change  of  variables  u * x - £^y  we  have 
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subject  to  x(l-£^)^u^x. 


V [x,i]  - »ax  E V 
u 


Note  this  set  is  a sublattice  in  the  pair  (x,u)  by  Example  A1.6 
and  Lemma  A.l. 


Now  consider  V x(l  + — | - tr-  u,j  ! . 

"_1L  v Ei/ 


For  R > 0 1 + -r—  ^ 0 


— — > 0 and  thus,  since  V _[x,j]  is  concave  in  x , 

— n-1 

i Jx  ^1  + ^ u,jj  is  superadditive  in  x and  u by  Example 


A2.5. 

For  R < 0 , we  need  only  consider  those  R satisfying 

- 1 <_  R <_  - 5^  , because  of  the  definition  of  . 

Then  — - 1 and  thus 

5i 


R 

, R 

= V , 

u - 

1 + — 

X,  j 

n-1 

Ci 

5i 

is  again  superadditive  in  x and  u . 

Since  mixtures  of  superadditive  functions  are  superadditive 
~e  have  + i)  is  superadditive  in  x and 

u , on  the  sublattice  x(l  - £ ) <_  u <_  x . Thus,  by  the  isotonicity 

theorem  (A. 11)  u (x,i)  , the  optimizing  value,  is  increasing  in  x . 
n 

This  optimal  value  is  clearly  u (x,i)  = x - £.y  (x,i)  . ■ 

n in 


Tlie  remark  that  follows  Theorem  3,1  clearly  applies  here  as  well. 
The  monotonicity  property  of  x - £^yn(x,i)  is  tight,  in  the 
same  sense  as  that  of  x + p.y  (x,i)  as  can  be  seen  from  the  example 


preceeding  this  theorem. 


40 


There  5i  - h . For  5 > , in  the  region  4 < x <_  8 

x - 5y^(x,i)  * x(l  - 25)  + 85  decreasing  in  x . 

It  is  interesting  to  note  that  the  proofs  of  Theorems  3.1 
and  3.2  do  not  depend  on  the  allowable  values  of  x and  y . 

Thus,  even  if  we  are  constrained  to  integer  fortunes  and  integer 
wagers,  the  results  will  still  be  valid. 

The  combination  of  Theorems  3.1  and  3.2  also  yields  the  follow- 
ing result,  useful  in  the  actual  computation  of  optimal  policies. 

Corollary  3.3: 

Assume  > 0 and  5^  exists  5^  > 0 . Then,  for  e > 0 

> 

fl  1 V*  + £,1)  " yn<*-i) 

Proof ; 

x + P.y  (x,i)  is  increasing  in  x so 
in 

x + e + P^Cx  + e,i)  1 x + p^Cx.i) 

implying 

yn(x  + e,i)  - yn(x,i)  > - . 

Similarly  x - 5^yn(x,i)  is  increasing  in  x so 

x + e - C.y„(x  + e , i)  > x - 5 y (x,i) 
in  — in 

implying 

j-  > y (x  + e , i)  - y (x,i) 

5^  — n n 

i 


r 


From  this  result 


Corollary  3.4 


Assume  p > 0 and  £ exists  and  £ > 0 . Then  y (x,i) 


is  uniformly  continuous  in  x 


The  specialization  of  Corollary  3.3  to  coin  tossing  games  will 


prove  useful  in  the  sequel,  and  is  interesting  in  its  own  right 
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CHAPTER  IV 

DEPENDENCE  OF  THE  OPTIMAL  STRATEGY  ON  THE  GAMBLE 

We  now  consider  the  effect  of  the  gamble  on  the  betting 
strategy. 

From  the  examples  of  Chapter  II,  it  is  clear  that  assuming 
the  random  variables  are  stochastically  increasing  in  i , does 
not  guarantee  that  yn(x,i)  is  increasing  in  i . In  fact,  the 
examples  show  that,  even  if  one  gambler  is  allowed  to  play  the 
same  game  over  all  his  horizon,  and  another  gambler  is  allowed  to 
play  a stochastically  better  game  for  the  same  number  of  periods, 
it  does  not  follow  that  the  second  gambler  should  wager  more,  if 
their  fortunes  are  equal. 

It  is  thus  evident  that  we  must  restrict  the  class  of  gambles 
considered,  in  order  to  obtain  meaningful  results.  In  the  class  of 
games  considered  by  Ferguson  [7] , the  gambler  faces  at  each  stage 
a coin  tossing  game,  with  probability  of  winning  p^  , known  to  him. 
However  at  each  stage  the  p^  is  drawn  from  some  distribution  F(p)  , 
independently  of  the  outcome  of  the  bet  and  of  the  previous  values 
of  p . The  questior  of  whether  one  should  bet  more  with  a higher 
p is  answered  affirmatively  in  the  following. 

Proposition  4.1: 

Let  y^ (x,p)  be  the  optimal  wager  when  you  have  n plays 
available,  your  fortune  is  x and  you  race  a coin  toss  with  proba- 
bility p of  winning.  Assume  at  each  stage  p is  independently 
drawn  from  a distribution  F(p)  . Then  yn(x,p)  is  increasing  in 
P • 


f 


■Mi 


■M  ■ p 
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Proof ; 

By  identifying  the  state  of  the  gambling  system,  i , with 
the  probability  of  winning,  we  have 

V [x,p]  = max  {pEV  [x  + y,T]  + (1  - p)EV  , [x  - y,T]} 
n _ _ n-i  n-i 

0<y<x 

where  T is  a random  variable  with  distribution  F . Since,  for 
fixed  x , the  set  {(y,p)  :0<^y<^x,0<^p<^l}  is  a sublattice, 
we  only  need  to  show  the  superadditivity  of  the  function  being 
maximized. 

Let 

4>(y,p)  = pEV  i [x  4 y,T]  + (1  - p)EV  [x  - y,T]  , 
n-i  n-i 

by  Theorem  A3,  we  have  to  show  that  for  e > 0 

<Ky,p  + e)  - 4> (y,p) 

0 

is  increasing  in  y , or  equivalently  4>  (y , P ) is  increasing  in  y 

dp 

<t>(y,p)  = + y*Ti  ■ ^n-i^  ~ y»Ti 

which  follows  froti  the  fact  that  V ,[x,p]  is  increasing  in  x . ■ 

n-i 

Another  interesting  question  that  arises  is  that  of  two  gamblers 
with  the  same  utility  function,  both  facing  a coin  tossing  game  in 
which  their  probabilities  of  winning  are  constant  over  their  horizon 
but  different  from  each  other.  It  is  intuitively  appealing  that  the 
one  with  the  higher  probability  of  winning  should  bet  more.  The 
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direct  proof  of  this  is,  however,  surprisingly  difficult,  and  our 

indirect  proof  yields  also  an  answer  to  another  question  of  interest. 

Suppose  that  n different  coin  tossing  games  are  available, 

with  probabilities  of  winning  p,,p0 . You  are  allowed  to 

l ^ n 

play  each  game  once,  but  can  choose  the  order  in  which  you  will  play 
them.  The  question  of  which  is  the  optimal  order  has  its  rather 
surprising  answer  in  the  following. 

Theorem  4.2: 

In  the  game  described  above,  the  value  of  the  optimal  expected 
utility  V^[x]  is  independent  of  the  order  in  which  the  p^ 
are  chosen. 

Proof : 

Assume  n = 2 . The  optimal  strategy  will  consist  of  the 
following  information 

a)  The  game  to  be  played  first 

b)  The  bet  to  be  placed  in  the  first  game,  say  y 

c)  The  bet  to  be  placed  in  the  second  game  if  we  win  the  first 
one,  say  yy 

d)  The  bet  to  be  placed  in  the  second  game  if  we  loose  the  first 
one,  say  y . 

We  will  prove  the  result  by  showing  that  choosing  p^  first  and 
then  continuing  optimally  yields  the  same  expected  utility  that  can 
be  obtained  by  choosing  first  and  then  continuing  optimally. 


Let  q±  = 1 - p±  . 

Assume  we  play  p^  first  and  P2  second.  Then,  conditioning 
on  the  outcome  of  the  tosses,  our  expected  utility  is 

PxP2vtx  + y + + P1^2V^X  + y ” yw^  + 

q;LP2v[x  - y + yj  + q1q2v[x  - y - yLl 

and  clearly  the  optimal  betting  strategy  is  determined  by  solving 
the  nonlinear  problem 

max  {p^vtx  + y + yy]  + P1q2vtx  + y - ywl  + 

y’VyL 

q;LP2vtx  - y + yL]  + q-^vtx  - y - yJJ 
subject  to  x >_  y >_  0 

x + y i yw  i 0 

x-y  >yL>  ° . 

* * * 

Let  y , yw  and  y^  be  the  optimal  strategy. 

Similarly,  if  we  decide  to  play  p2  first  and  then  p.^  , we 
have  to  solve 

max  {p2P1V[x  + y + yw]  + P2q1V[x  + y - ywJ  + 

y ,yW,yL 

q2PlV[x  “ y + yJ  + q2qlVtx  “ y " yL)} 
subject  to  x >_  y >_  0 

x + y i y„  I 0 
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Define 


(4.1) 


y+  * i ( yw  + y* ) 

yw  = y +2(yw-yL) 

yL  = y ' 2lyW-yJ 


We  will  show  that  these  values  are  feasible  for  the  second  problem, 
and  yield  the  same  value  of  the  objective  function  as  was  optimal 
for  the  first  problem.  Hence  the  optimal  value  of  the  second  problem 
is  greater  than  that  of  the  first,  and  playing  P2  first  is  at  least 
as  good  as  playing  p^  first.  Reversing  the  argument  (since  we 
have  not  imposed  conditions  on  the  p^)  shows  both  maxima  are  the 
same  and  hence  the  order  in  which  the  p^  are  chosen  is  irrelevant. 

Feasibility  of  y , and  y^  : 


yW  - ° ’ yL  - 0 =*>  y+  - 0 

**  **  l/**\+ 

x + y 1 yW  * x - y iyL  x^2  lyW  + yJ  = y 

X + y - yw  “ x + y (yW  + yL  ) ' y "2(yW-yL) 

* * 

* x - y + yL  >_  0 

+ + 1 / * * \ * 1 / * * \ 

x.y  -yL-^-Y(y„  + yLJ-y  + y ( yw  - y,, ) 


* * 

x - y - yT  l o 


Also  y^  is  the  amount  to  bet  when  there  is  one  stage  left 

H ic 

and  our  fortune  is  x + y (since  we  have  won  our  bet  of  y 
is  the  previous  stage) . Hence 
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y;  = yi(x  + y**p2)  • 

Similarly  y*  = y^x  - y*,P2)  • 

Thus  by  Corollary  3.5 

* * . * 

- yJ  i 2y 

showing  y*  * 2 VW  ~ yi.)  i,e'  yW-0,yL-°‘ 

Hence  y+  , y+  , y+  satisfy  the  constraints  of  the  second 

problem.  By  substituting  them  in  its  objective  function  one  obtains 

p2piv[x  + y+  + y^]+  p2qiv[x  + - 4] + 

q2PlV[X  " y+  + yL  J + q2qlV[x  - y+  - yt]  = 

PlP2v[x  + y*  + y*]  + p^fx  - y*  + y*]  + 

q2plV[x  + y*  - yw]  + q2qlV[x  - y*  - y2] 

which  was  the  optimal  value  in  the  first  problem.  It  now  remains 
only  to  complete  the  proof  for  n :>  2 . 

Suppose  in  an  (n  - 1)  stage  problem  the  order  of  the  p^  is 
irrelevant.  Consider  an  n stage  problem.  Choose  p^  to  be  the 
first  and  then  continue  optimally  with  p2,p2>  ...,  p^  . This  is 
equivalent  to  choosing  p^  and  playing  a one  stage  game  where  the 
final  utility  is  that  given  by  the  optimal  value  function  of  the 
n - 1 stage  problem  with  probabilities  p2,p^,  . ..,  p^  . 

Thus  we  can  choose  any  one  of  these,  say  p2  , to  be  second, 
and  continue  with  p^.P^*  •••*  Pn  • 
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But  the  first  two  stages  can  be  thought  of  as  a two  stage 
problem  whose  final  utility  is  the  optimal  value  function  of  the 
n - 2 stage  game  with  p^,p^,  . ..,  p^  . Thus  in  this  two  stage 
game  we  can  change  the  order  and  play  p^  first  then  p^  without 
affecting  the  overall  optimal  value.  Thus  choosing  P2  first  is 
at  least  as  good  as  choosing  p^  first  and  since  the  numbering  of  the 
p^  was  arbitrary  the  result  follows.® 

Now  consider  a gambler  who  faces  n tosses  of  the  same  biased 

coin,  with  probability  p > h of  winning. 

Let  y (x,p)  be  his  optimal  wager.  We  will  study  the  dependence 
n 

of  yn(x,p)  on  p . 

First  we  have  an  interesting  functional  equation  satisfied  by 
y^(x,p)  , which  says  that  the  optimal  wager  is  the  average  of  the 
optimal  wagers  to  be  placed  upon  loosing  and  upon  winning. 

Proposition  4.3: 

yp (x,p)  = y {yn_1(x  + yn(x,p),p)  + yn_L(x  - yn(x,p),p)}  . 

Proof ; 

In  the  game  of  Theorem  4.2  set  p^  = P2  = = * * * Pp  “ P • 

Then  exchanging  p^  and  P2  does  not  change  the  optimal  policy. 

+ * 

Hence  y * y and  Equations  (4.1)  apply.  Noting  that 

'it  it 

y = yn(x,p)  , yy  = y^Cx  + yn(x,p),p)  . yL  = yn_1(x  - yn(x,p),p)  , 
the  result  follows  from  the  first  of  these  equations.® 

An  interesting  property  of  this  coin  tossing  game  is  the 


1 


following . 


Lemma  4.4: 

Assume  that  for  some  x > 0 y^Cx.p)  ■ x . If  V[ • ] is 
strictly  increasing  and  oif ferentiable,  then,  for  all  x'  e [0,x] 

yn^X' ,P^  “ x'  • 

Proof : 

By  Theorem  1.6  V^[«,p]  is  strictly  increasing  and  differentiable 

By  Theorem  1.2  it  is  concave. 
y^(x,p)  is  determined  by 

max  {pV  [x  + y ,p]  + (1  - p)V  [x  - y,p]}  . 

0<y<x  n_i  n*i 

By  the  concavity  of  V^^O.p]  > yn(x>P)  r'  x ^ an<^  only  if 

pV'  [2x,p]  - (1  - p)V'  [0,p]  > 0 . 
n-i  n-x 

Now,  for  x'  e [0,x] 

pvVfUx'.p]  - (i  - p)v;_1[o,p]  ipv;_1[2x,P]  - (1  - pJV^O.p]  >0 

since  V'  ,[•,•]  is  decreasing  in  the  first  argument  by  concavity. 
n-1 

Hence  yn(x',p)  = x'  . ■ 

We  can  now  prove 
Theorem  4.5: 

Assume  V is  strictly  increasing  and  differentiable.  Then 
yn(x,p)  is  increasing  in  p , and  yn(x,p)  < x implies  yn(x,p) 
y (x,p  + 6)  for  0 < 6 < 1 - p . 


< 


50 

Proof ; 

We  first  show  that  these  properties  hold  for  n « 1 . y^Cx.p) 
is  determined  by 

max  {pV (x  + y)  + qV(x  - y)}  . 

0<y<x 

If  y^ (x,p)  = x then,  since  V is  differentiable 

pV'  (2x)  - qV'  (0)  >_  0 . 

But  then,  for  e > 0 

(p  + e)V'(2x)  - (q  - e)V'(O)  = PV'(2x)  - qV' (0)  + e[V' (2x)  + V' (0)]  > 0 

hence  with  p + e we  also  bet  x . 

If  0 < y^x.p)  < x then 

pV'[x  + y^x.p)]  - qV'[x  - y^x.p)]  - 0 
implying  for  e > 0 

(P  + e)V'[x  + y^x.p)]  - (q  - e)V'[x  - y^x.p)]  «= 
pV'[x  + y1(x,p)]  - qV'[x  - y1(x,p)]  + 
e {V'[x  + y^x.p)]  + V’[x  - y1(x,p)]}  » 

e {V’[x  + y-^x.p)]  + V'[x  - y1(x,p)]}  > 0 

since  V(»)  is  strictly  increasing.  Hence 


y1(x,p  + e)  > y^x.p)  . 


■ 
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The  case  y^(x,p)  * 0 does  not  arise  since 


pV'[x]  - qV'[x]  = (p  - q)V'[x]  > 0 

because  p > % and  V(*)  is  strictly  increasing.  For  n > 1 . 
Choose  p > h and  pick  p+  = p + e > p . 

Let  y = yn(x,p)  and  y = y^(x,p  ) . Assume,  as  an  inductive 
hypothesis,  that  y^^Cx.p)  satisfies  the  properties  we  are  trying 
to  establish.  Proposition  4.3  states  that 

y * \ {yn-ltx  + y,p]  + yn-ltx  ■ y,p] } 

— 1 / r — H +■.  r — H +i  ) 

y = j \yn_i[x  + y *p  1 + yn_i[x  ‘y-pl)' 
y - y+  - \ {yn_i(x  + y.p)  + yn_i(x  ■ y»p>  ~ yn-i(x  + ^ ,p+)  " 

Vi(x  - ? •p+)}  • 


If  y , (x  + y,p)  < x + y (implying  y < x)  then  since 
n-l 

y^  ^(x  + y,p)  is  strictly  increasing  in  p 

y - y+  < \ {yn_i (x  + y,p+)  + y^C*  ~ y«P+)  ~ 
yn-1  (x  + y+,p+)  - yn_x(x  - ^.p*)}  • 

Now  Corollary  3.5  states 

yn-1(x  + y,p+)  - yn_x(*  + y4 »p+)  i ly  ~ y+l 


yn_L(x  - y,P+)  - yn_L(x  - y^.p'1')  <.  |y  - y 


-+  +v 


Hence  y - y"’’  < jy  - y+|  implying  y - y"1"  < 0 
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If  yn_]/x  + y+,P+)  K x + y*  (implying  y+  < x)  then  since 
yn-1 (x  + y+,p+)  is  strictly  increasing  in  p+ 

- -+  1 < 

y - y < 2Vn-lix  + y,p)  + yn-l(x  " y,p)  " 
yu-i(x  + *+ ,p)  " yn-i(x  " y+*p^} 


and  the  same  argument  applies,  showing  y < y . In  the  case 

— _ -f- 

yn_^(x  + y,p)  = x + y and  yR  ^(x  + y »P  ) =x+y  then,  by 
Lemma  4.4  y^ (x  - y,p)  = x - y and  y X(x  - y+,p+)  = x - y+ 
Hence  y = x y+  = x , completing  the  proof.  ■ 


■ 
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CHAPTER  V 


GENERALIZATION  TO  ADAPTIVE  MODELS 


In  many  gambling  situations  the  player  does  not  know  the  exact 
distribution  of  the  random  variables  that  determine  the  outcome  of 
the  game.  A prime  example  are  the  slot  machines  of  Nevada  casinos, 
where  the  probabilities  of  the  different  possible  outcomes  are  a 
closely  guarded  secret.  In  such  cases,  the  best  the  gambler  can 
hope  for  is  to  obtain  information  on  the  distribution  as  the  game 
progresses.  This  accumulation  of  information  and  subsequent  modi- 
fication of  behaviour  is  usually  termed  adaptive  behaviour. 

We  will  assume  initially  a fairly  general  scenario,  which  will 
later  be  restricted  to  coin  tossing  games. 

Assume  a gambler  with  an  initial  fortune  x faces  n realizations 
of  a game.  If  he  bets  an  amount  y e [C,x]  his  fortune  will  change 
to  x + Ry  where  R is  a random  variable.  He  knows  also  that  the 
distribution  of  R is  the  same  throughout  all  n plays.  Furthermore 
he  knows  the  distribution  is  a member  of  the  set  {Gt  : t ^ 0}  , 
although  he  does  not  know  with  certainty  which  one  ■ft  is. 

From  past  experience  he  has  formulated  a prior  distribution 
F(*)  with  the  interpretation  that,  if  G^  is  the  distribution  he 
will  be  using  then 


F(t)  = P[T  < t] 


After  observing  the  outcome  of  the  first  play,  the  gambler  computes 
in  the  usual  manner,  a posterior  distribution  on  T , given  by  the 


conditional  distribution 
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Fr(t)  - P[T  £ t/R  - r] 


and  he  then  continues  the  game,  with  a new  fortune  and  a new  prior 
distribution. 

We  will  assume  that  the  gambler  wishes  to  play  in  such  a way 
as  to  maximize  the  expected  value  of  the  utility  V , of  his  final 
fortune,  where  his  utility  function  possesses  the  properties  dis- 
cussed in  Chapter  I. 

We  will  also  assume  that  there  exists  an  upper  bound  on  the 
possible  outcomes  of  the  gamble,  that  is,  we  will  assume  there 
exists  a 0 , 1 < 6 < <=  such  that  Gt(0)  = 1 for  all  t . 

If  the  gambler  decides  not  to  bet  at  any  stage  of  the  game, 
then  we  will  assume  that  the  game  is  "played"  nevertheless,  and  the 
gambler  observes  the  outcome  of  the  game,  thus  gaining  informatior 
without  risk. 

Let  V^tx.F]  be  the  supremum  expected  utility  for  a gambler 
whc  starts  out  with  a fortune  of  x , a prior  distribution  F and 
n stages  to  go. 

Then  the  usual  dynamic  programming  formalism  shows  that 


V",X'F1  'oSJ/V»-1 


[x  + ry,Fr]dG(r) 


Vq[x,F]  = V[x] 


where 
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G ■ J" GtdF(t)  , 

0 

and  F is  as  defined  above. 

K 

We  must  now  proceed  to  prove  the  existence  of  an  optimal 
policy,  and  the  properties  of  the  optimal  value  function  V^[x,F]  . 

Lemma  5.1: 

If  V ,[x,F]  is  continuous  in  x on  (O,00)  and  right 
n— l 

continuous  at  the  origin  for  all  F , then  there  exists  a number 
yn(x,F)  such  that 

00 

v [x,F]  = f V [x  + ry  (x,F),F JdG(r)  . 
n J n-i  n ~ 

-1 

Theorem  5.2: 

For  all  F , V [x,F]  is  increasing  and  concave  in  x . It 
n 

is  thus  continuous  in  x on  (0,<*>)  . Furthermore  Vn[x,F]  <_ 

V[x(l  + 9)n]  , and  it  is  right  continuous  at  the  origin.  Hence  an 
optimal  policy  exists. 

From  this  theorem  it  follows  that  the  supremum  in  the  optimality 
equation  is  always  attained,  and  we  can,  and  will  write  it  as 


V [x,F]  = max  /V  [x  + ry,F JdG(r) 
n A.  . j n-  x f 

0 <y <x  J i 


VqIx.F]  = V[x]  . 


We  will  call  yn(x,F)  the  optimal  amount  to  wager  and  if 
not  unique  we  will  assume  that  either  the  smallest  or  the  largest 
is  chosen,  as  long  as  the  gambler  is  consistent  in  his  choice. 

The  proof  of  the  above  is  identical  to  that  of  Lemma  1.1  and 
Theorem  1.2,  except  that,  in  this  case,  due  to  the  definition  of 
6 , it  is  immediate  that 

V [x,F]  < V[x(l  + e)n] 
n — 

since,  as  you  cannot  win  more  than  0 in  any  given  play,  the  best  you 
can  hope  for  is  to  win  all  games,  and  to  have  wagered  all  your  fortune 
at  each  toss,  and  your  fortune  will  then  be  smaller  than  x(l  + 6)n  . 

The  proof  of  Proposition  1.3  shows 

Proposition  5.3; 

V^tx.F]  is  increasing  in  n . 

A rather  surprising  fact  is  that,  in  the  adaptive  model,  it  is 
sometimes  optimal  to  bet  a positive  amount  in  unfavourable  games, 
as  the  following  example  shows. 

Consider  a coin  tossing  game.  The  probability  of  winning  is  known 
to  be  either  1 or  0 with  a prior  probability  .4  of  being  1.  One  desires 
to  maximize  his  expected  fortune  (i.a.  V[x]  = x)  in  two  stages. 

It  is  easy  to  see  that  the  optimal  strategy  is:  bet  x . If  you 

win,  bet  2x  . If  you  lose  bet  nothing. 

Thus,  although  in  the  first  toss  the  probability  of  winning  is 
.4,  one  should  wager  all  his  fortune. 

xe 

The  special  utilities  V[x]  = Ln(x)  and  V[x]  = — 0 < 1 0^0 

P 

yield  simple  strategies. 


Theorem  5.4: 


If  V[x]  *•  Lnx  then 

V [x,F]  = V [1,F]  + Lnx 
n n 

yn[x,F]  « xa(F) 

where  a(F)  does  not  depend  on  n or  x . 

B 

If  V[x]  B < 1 , B ^ 0 then 

P 

V [x,F]  = V [l,F]xe 
n n 

yntx,F]  = xon(F) 

where  an(F)  does  not  depend  on  x . 

The  proof  of  this  theorem  is  identical  to  that  of  Theorems  2.1 
and  2.2,  and  is  hence  omitted. 

The  dependence  of  y^Cx.F)  on  x is  identical  to  that  in  the 
nonadaptive  case. 

As  defi  ed  in  Chapter  III,  let 

p = sup  {r  : r > 0,G(r)  = G(0)} 

F 

5 = | inf  {r  : r < 0,G(0)  ■ G(r)}| 

r ' 

where  G = J*GtdF(t)  . 

Theorem  5.5: 

Let  yn(x,F)  be  the  optimal  amount  to  wager.  If  pp  > 0 

then  x + p„y  (x,F)  is  increasing  in  x . If  > 0 then 
rn  i 


x - £Fyn(x,F)  is  increasing  in  x . 
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f 


The  proof  of  this  theorem  is  identical  to  that  of  Theorems  3.1 
and  3.2  and  is  hence  omitted. 

From  this  theorem  there  immediately  follows 


Corollary  5.6: 

If  both  p > 0 and  £ > 0 then,  tor  e > 0 

r r 


fiy  (x  + e,F)  - y (x,F)  > - 
— n n — 


e 


hence  y^Cx.F)  is  uniformly  continuous  in  x . 

Corollary  5.7: 

In  coin  tossing  games  |y^(x  + £**')  ~ (x , F)  | <_  t . 

The  proofs  are  of  course  repetition  of  those  of  Corollaries  3.3, 
3.4,  3.5. 

We  now  turn  to  an  examination  of  the  dependence  of  both 

V (x,F)  and  y (x,F)  on  the  prior  distribution  F . As  in  the  case 
n n 

of  nonadaptive  models,  we  will  limit  ourselves  to  coin  tossing  games. 

In  this  case,  each  distribution  , i = 1*2,  ...  is 
characterized  by  a probability  p^  of  winning.  The  prior  distribution 
F is  characterized  by  a sequence  ...}  with  the  interpreta- 

tion that  the  game  we  are  playing  will  have  probability  p^  of  a 
win  with  probability  , that  is,  we  will  be  playing  against  coin  i 
with  probability  it  . 

Let  V^tx.II]  be  the  optimal  expected  utility  for  an  n stage 
game  when  our  initial  fortune  is  x and  we  have  a prior  probability 
distribution  11  . 


I 

j 

i 

i 

i 

i 

I 


A 


Theorem  5.8 


Let  V [x,n,6(x)]  be  the  expected  utility  in  an  n stage  game 


our  prior  distribution  is  II  and  we  use 


when  our  fortune  is  x 


Clearly 


Let  B [x,p,6(x)]  be  the  expected  utility  in  an  n stage  game 


when  our  initial  fortune  is  x , the  probability  of  winning  is  p 


Now  consider  two  prior  distributions  II  and  II  and  any  real 


numbers 
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Thus  proving  the  convexity  of  V^lx.II]  in  h . ■ 

In  order  to  analyze  monotonicity  properties  of  V^tx.H]  in  II  it 
is  necessary  to  impose  some  ordering  on  the  collection  of  allowable 
n's  . 

We  will  assume  that  the  possible  values  of  the  probability  of 
winning,  p^  , are  ordered  so  that  < 1*2  < ***  * 

Let  H = t tt^ , , ...)  and  ft  = {w^.w^,  ...}  be  two  prior  densi- 
ties. We  will  say  II  (strictly)  dominates  ft  (n  ^ ft  or  II  >-  ft)  if 
and  only  if  for  all  i and  j i < j such  that  both  mass  functions  do 

not  vanish  identically  we  have  — <_  — (or  with  strict  inequality) . 

Wi  Wj 

This  definition  is  the  usual  Monotone  Likelihood  Ratio. 

It  is  well  known  that  if  II  (strictly)  dominates  ft  then  H 
is  stochastically  larger  (strictly)  than  ft  . Hence  it  follows 


I P 


i^i 


(or  with  strict  inequality)  . 


If  II  is  a prior  density,  we  will  call  W(II)  and  L(H)  the 
posterior  densities  given  we  have  won  a game  and  that  we  have  lost 


one  respectively. 
Clearly 
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We  will  need  the  following. 

Proposition  5.8; 

Let  n and  ft  be  two  prior  densities.  Assume  II  > ft  then. 

a)  W(IT)  ^ n £ L(R) 

b)  W(II)  ► W(ft) 

c)  L(n)  L(ft)  . 

If  n >■  ft  then  b and  c are  strict. 

Proof : 

To  show  a)  we  form  the  ratios 


Vi 


I V, 


?Vj 


and 


1 - 1 Vi 

i - pt 


which  are  clearly  increasing  in  i since  p^  is  . 
To  show  b)  and  c)  we  form  the  ratios 

■Ti  / p±wi  l pi“i  *i 
l Pj'j  / I Pj-j  l Pj'j  “i 

and 


A 


V . 
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U - Pi)^i  / (1  “ Pi)wi  1 - [ P^.  ir± 

i - 1 p^  / 1 - 1 Pj»j  1 - l pj'i  "i 


which  are  clearly  increasing  in  i since  — is  . The  proof  in 

Wi 

the  strict  case  is  of  course  identical.® 


We  are  now  in  a position  to  prove  that  the  ordering  of  the 
prior  densities  defined  above  induces  an  ordering  of  the  optimal 
value  function. 


Theorem  5.9: 


Let  II  and  n be  two  prior  densities.  If  II  > fi  , then  for 

all  n and  x V [x,II]  > V [x,ft]  . 

n — n 


Proof : 


The  proof  proceeds  by  induction  on  n . For  n = 0 
Vq[x,II]  = V[x]  * Vgtx.fi]  for  all  x , H and  n . Assume 

1tx,n]  >_  v^_^[x,ft]  for  x • n and  ^ such  that  n ^ n 

Vn[x,n]  = max  l(l  p^  W ^[x  + y,W(n)]  + 

0<y<x  ' ' 


(x  - I P^JViI*  " y,LCn>]  | 


By  Proposition  5.8  W(II)  >_  W(ft)  and  L. (II ) ^ L(ft)  . Hence 
by  the  induction  hypothesis 


63 


V U.nl  >_  max  p.ir.)v  [x  + y,W (fi)]  + 

0<y<x  ,V  1 ' n-i 

(i  - I Pi^i)vn.itx  ~ y»LWj} 

>_  max  PiO^n-l^  + y»W^^  ~ 

V [x  - y,L(fi)])  + V fx  - y,L(.Q)]l  . 
n—  i n— x j 

By  Theorem  5.2  ^[x,II]  is  increasing  in  x . Thus, 

V [x  + y,W(n)]  - V [x-y,L(fi)]  > V [x-y,W(Q)]  - V [x-y,L(3)] 
n-l  n-x  — n-i  n— x 

and,  since,  by  Proposition  5.3  W(fi)  L(S2)  , we  have  by  the  induction 
hypothesis 

V . [x  + y,W(fl)]  - V -|  I x — y,L(£2)]  > 0 . 
n— l n— x — 

Thus,  since  \ p^ir  >_  \ p.^  , we  have 


vn[x,n]  _>  max  PiWi)^Vn-l^x  + y»W^l  " Vn-l^x  " y»L(fi)D  + 

0<y<x 

Vn-l[x  * y>L(f2>l} 


which  is 


V [x,n]  > V [x.fl] 
n — n 


completing  the  induction. ■ 

The  question  now  arises  as  to  the  dependence  of  the  optimal 
wager  yn(x,H)  on  the  density  II  . One  would  wish  to  show  that  if 
II  > !!  then  yn(x,H)  y^Cxft)  . 
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The  proof  of  this  monotonicity  requires  however,  as  in  the  non- 

adaptive  case,  an  indirect  approach,  based  on  examining  games  in 

which  different  gambles  are  available  and  showing  the  irrelevance 

of  the  order  in  which  the  gambles  are  chosen. 

Suppose  a player  faces  n coin  tosses,  with  different  coins. 

He  can  choose  the  order  in  which  he  plays  the  coins  but  each  coin  can 

only  be  used  once.  His  gambling  universe  is  in  some  state  i e 

{1,2,  ...}  unknown  except  for  a prior  distribution.  Let  = 

P[state  of  the  universe  is  i]  . If  the  state  of  the  universe  is  i , 

and  he  chooses  coin  j , he  will  win  with  proOability  p^  and  loose 

with  probability  = 1 - p_.^  . The  state  of  the  universe  remains 

fixed  throughout  the  game,  but  is  unknown  (except  for  the  prior 

distribution)  to  the  gambler,  who  gains  information  solely  on  the 

basis  of  the  outcome  of  his  bets.  As  usual  the  gambler  acts  so  as 

to  maximize  the  expected  value  of  his  utility  function  V [ • ] . 

Let  V [x,n]  be  the  expected  utility  in  an  n stage  game  if 

one  starts  with  a fortune  of  x and  a prior  distribution  II  and 

proceeds  optimally.  Let  y [x,H]  be  the  optimal  wager  in  this  case. 

n 

Note  the  dependence  of  these  two  functions  on  the  values  and  order 
of  the  p_.^  which  has  been  suppressed  for  simplicity  of  notation. 

We  will  show  that,  under  some  conditions,  the  order  in  which  the 
coins  are  chosen  is  irrelevant. 

Theorem  5.10: 


In  the  game  described  above,  assume  p^  > h for  all  i and  j 
Further  assume  V [ • ] is  strictly  increasing,  differentiable  and 
strictly  concave. 


F 
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Then  V^tx.II]  is  independent  of  the  order  in  which  the  coins 

are  chosen,  and  for  all  possible  orders  y (x,II)  > 0 for  all  x 

n 

and  II  . 

Proof: 

The  proof  proceeds  by  induction  on  n . 

V [x,n]  clearly  does  not  depend  on  the  order  of  the  coins, 
since  there  is  only  one  coin.  y^(x,Il)  is  chosen  by  solving 

^max  pii7ri)vtx  + y}  + (l  ~ 

by  the  differentiability  of  V we  obtain  at  y = 0 

l (P1±  " <lli)1TiV,[x]  > 0 

since  V is  strictly  increasing  and  p^  > q for  all  i . 

By  the  strict  concavity  and  smoothness  properties  of  V this 
implies  y^(x,JT)  > 0 . 

Now  assume  that  in  any  n - 1 stage  V^_^[x,H]  does  not  depend 
on  the  order  of  the  coins,  and  for  any  order  chosen  y^  ^(x,II)  > 0 . 

Now  consider  an  n stage  game  where  coins  1,2,  ...,  n are 
available.  Assume  we  pick  coin  1 to  be  played  first.  Whatever  the 
outcome  of  the  game,  we  will  be  faced  with  an  n - 1 coin  game 
with  coins  2,3,  ...,  n . By  the  induction  hypothesis  any  of  these 
can  be  chosen  to  continue.  Assume  the  gambler  chooses  coin  2 
whether  or  not  he  wins  on  the  first  toss.  We  will  prove  that  the 
gambler  could  have  done  just  as  well  by  choosing  coin  2 as  the  first 


& 


- - ■ . nauBfrinii.  1 1 


— , — .. 


coin.  If  the  gambler  chooses  coin  1 and  then  coin  2,  his  optimal 
expected  utility  is  given  by 

max  {vn_2Lx  + y + y^V^Ol)]  £ + 


y,yw,yL 


Vn-2[x  + y ' yw.L2WL(n)]  l Pliq2i1ri  + 
Vn-2[X  " y + VW2L1(II)]  l ^liP2i^i  + 
Vn-2[X  ' y ' yL,L2Ll(11)]  l ^li^i’i} 


subject  to  y ^ 0 


x > y 


yw  L 0 x + y i yw 

yL  1 0 x - y i yL 


where  y is  the  optimal  wager  in  the  first  stage,  y and  yT 
are  the  optimal  wagers  in  the  second  stage  if  one  wins  or  loses 
respectively  in  the  first  stage.  1^(11)  and  1^(11)  represent 
the  posterior  distribution  if  one  respectively  wins  or  loses  in 
a loss  of  coin  i and  the  prior  distribution  is  II  . 

Clearly 


w2wx(n) 


wxw2(n) 


LjWjOi)  - w^on 


w2Lx(n) 


L1w2(n) 


L^Ol)  = LxL2(n) 


it  if  ^ 

Let  y , » yL  the  optimal  values  of  the  variables  in 

this  optimization  problem.  They  are  unique  since  we  are  maximizing 


a strictly  concave  function  over  a compact  set.  Clearly  by  the 
induction  hypothesis 


y*  = yn(x,n) 

(5.1)  y*  = yn-1[x  + y^V^fll)]  > 0 

= yr,  “ y*»Li  > 0 ' 

L n— 1 1 

* k 

Since  we  know  that  in  the  optimal  solution  yw  > 0 and  y^  > 0 , 
the  nonnegativity  constraints  on  y^  and  y^  can  be  relaxed 
because  of  the  strict  concavity  of  the  objective  function  and  the 
convexity  of  the  constraint  set.  However  we  need  to  guarantee  the 
nonnegativity  of  the  first  arguments  of  vn  2 ’ an<*  hence  t*ie 
nonnegativity  constraints  may  be  replaced  by 

x + y + yw>^0  x-y  + yL^_0. 

Now  consider  playing  coin  2 first,  followed  by  coin  1. 

The  optimal  strategy  is  given  by 

maX  (Vn-2[X  + y + yW’WlW2(II)]  l p2iPli1Ti  + 
y ,yW’yL 

Vn-2[x  + y ~ yW’LlVn)]  ^ P2iq2i*i  + 

Vn-2[x  " y + yL»WiL2(II)]  2 q2ipli1Ti  + 

vn-2[x  - y ’ yL’LlL2(I1)]  l q2iqli1Ti } 

subject  to  y ^ 0 x + y >_  yw  x + y + yw  0 

x>_y  x-y>_yL  x-y  + yL>_0. 
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Since  again  the  nonnegativity  constraints  on  yw  and  y^  may 
be  relaxed. 

Define 


(5.2) 


+ 1 / * . * \ 
y =2lyw  + yL) 

+ * 1 / * *\ 

yw  = y + Y \ y W ~ yL  / 

+ * 1 / * * \ 

yL  = y ~ T \ y W ~ yL  / 


We  now  proceed  to  verify  that  these  satisfy  the  constraints  of  the 
above  maximization  problem 


A A + 

y > 0 and  y > 0 imply  y > 0 
W Li 

AA  AA  ]/AA\  + 

x + y >_  yy  and  x - y >_  yL  imply 

+ + AA  ++  AA 

x+y  -yw=x-y  +yL>_0;x  + y +yw=x  + y +yw>  0 

+ + AA  +4-  AA 

x-y  - yL  = X - y -yL>.0;x-y  +yL«=x  + y - yw  > 0 


Thus  y+  , y^  , y^"  are  feasible  for  this  second  maximization 
problem.  Direct  substitution  in  the  objective  function  shows  that 
they  yield  the  same  value  of  the  objective  function  as  was  optimal 
for  the  first  problem.  Thus  playing  coin  2 first  and  then  coin  1 
is  at  least  as  good  as  playing  coin  1 and  then  coin  2.  Since  the 
numbering  of  the  coins  was  arbitrary,  reversal  of  the  argument 
shows  that  playing  coin  1 first  is  at  least  as  good  as  playing  coin 
1 first.  Hence  both  alternatives  are  equivalent. 
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e 


y , y are  optimal  for  the  second  problem. 
W L 


Also , y 

However,  by  the  strict  concavity  of  the  objective  function  in 
this  problem,  these  must  be  the  unique  solution  to  the  maximization 
problem.  Thus,  since 


y+-'Hyw  + y£)>  0 
y*‘HyI  + yL)>  ° 


we  have  that  y (x,II)  > 0 completing  the  induction, 
n 


The  question  of  whether  one  can  drop  the  assumption  that 


p . > \ is  answered  in  the  negative  in  the  following  counter 


example. 

There  are  two  states  of  the  universe  and  two  coins.  Coin  1 


has  p.^  = 1 , p12  = 0 . Coin  2 has  p21  * 0 , p22  = 1 . The 


3 1 

prior  distribution  is  an<*  t*ie  utility  function 


is  Lnx  . 

Choosing  coin  1 first  and  coin  2 second 


V2[x,H]  = Lnx  + .3041  , y^x.H)  * y x . 


Choosing  coin  2 first  and  coin  1 second 


V2[x,n]  = Lnx  + .5199  , y2(x,II)  = 0 


Thus  one  should  choose  coin  2 first  and  then  coin  1.  The 


reason  for  the  failure  is  of  course  that  with  some  p^  < h we 


find  it  desirable  to  bet  0.  Hence  y (x,H)  = 0 and  the  non- 


negativity constraint  may  not  be  dropped  as  we  did  in  our  proof. 
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Now  consider  a gambler  who  faces  n tosses  of  the  same  coin. 

He  does  not  know  the  probability  of  winning,  except  that  it  is  one 

of  p^  < Pj  < P3  < * • • . He  knows  a prior  density  II  = [ , ^2 » •••] 

with  the  interpretation  ir^  ■ P[the  coin  is  of  type  i]  . We  wish 

to  study  the  dependence  of  his  optimal  wager  y (x,n)  on  II  . 

n 

We  first  give  a proposition,  useful  in  its  own  right,  that  gives 
a recurrence  relation  satisfied  by  y^Cx.II)  . 

Proposition  5.11: 

Assume  p^  > h for  all  i . Further  assume  V[*]  is  strictly 
increasing,  differentiable  ar.d  strictly  concave.  Then 

y (x,n)  > 0 
n 

and 

y (x,n)  = \ {y  . [x  + y (x,n)  ,W(I1)]  + y [x  - y (x.II)  ,L(n)] } . 
n l n-X  n n— 1 n 

Proof : 

In  the  game  of  Theorem  5.10,  set  p^  * Pi  for  j = 1,2,  ....  n 

Then,  exchanging  coins  1 and  2 does  not  change  the  optimal  policy. 

+ * 

Hence  y = y . Also  Equation  (5.2)  apply.  These,  together  with 
(5.1)  imply  the  desired  results.® 

Theorem  5.12: 

Let  II  and  ft  be  two  prior  distributions.  Assume  p^  > H 
for  all  i . Further,  assume  V[*]  is  strictly  increasing, 
differentiable  and  strictly  concave.  If  II  strictly  dominates  ft  , 
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then  for  all  n and  x y (x,II)  y (x,ft)  . Furthermore  if 


y (x,Q)  < x , then  y (x,II)  > y (x,ft)  . 
n n n 


Proof : 


Our  proof  proceeds  by  induction.  For  n = 1 , y^Cx.ft)  is 


given  by 


^max  | V[x  + y]  £ p^  + V[x  - y]  (l  - l Piwi)|  * 


If  y^(x,ft)  = x , then  we  must  have 


V'[2x]  l piwi  > V-[0](l  - l pjWj'J 


Since  £ p^r^  > I piw±  anc*  V strictly  increasing 


V’t2x]  l piTTi  > V’[2x]  l p.w.  > v’[0](l  - l p.„J  > V'[0](l  - [ p^J 


implying  y^x.II)  = x 


If  0 < y^(x,fi)  < x then  we  must  have 


V'[x  + y^x.fi)]  l p±wi  = v'[x  - y;L(x,n)](l  - l . 


The  same  argument  shows  thus  that 


V'[x  + y^x.fl)]  l pi7ri  - V'[x  - y(x,fl)]^l  - J ^ > 0 


implying  y^x.II)  > y^x.ft) 


By  Proposition  5.11,  the  case  y^Cx.ft)  ■ 0 does  not  arise. 


+ * 

To  simplify  notation,  let  y = yn(x,ft)  , y = y^Cx,!!)  . 


1 
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Assume  the  theorem  is  true  for  n - 1 . Then  by  Proposition  5.11 


(5.3) 


Then 


(5.4) 


y+  = \ {yn_2 [x  + y+,W(ft)]  + yn_^ [x  - y+,L(fi)]| 

y*  = \ {yn-i^x  + y**w(n>l  + yn-i^x  ~ y*»L(n)l}  • 

y+  - y*  = y jy^-Jx  + y+,W(fl)]  + yn-1fx  - y+,L(fi)]  - 
y^tx  + y ,w(n)]  - yn-1[x  - y*,L(n)]j-  . 


If 


+ 

y < x 


then  by  5.3,  either 

y^fx  + y+,W(fi)]  < X + y+ 


or 


y i [x  ~ y+,L(n)]  < x - y+ 

n-x 

or  both. 

By  the  induction  hypothesis,  in  the  first  case 

yn-1[x  + y+,w(n)]  < yn-1[x  + y+,w(n)] 

and  in  the  second  case 


y ,[x  - y+,L(fi)]  < y [x  - y+,L(JI)]  . 
n-i  n— x 


In  either  case  5.4  can  be  written  as 


y+  - y < j {yn_x [x  + y+,w(n)]  - yn-1tx  + y ,w(n)]  + 
y^fx  - y+,L(n)]  - yn-1tx  - y*,L(n)]|  . 

By  Corollary  5.7 

y . [x  + y+,W(II)]  - y [x  + y ,W(II)] 
n-x  n— X 

y [x  - y+,L(II) ] - y [x  - y*,L(n)] 
n-X  n-x 

Hence 

+ * i + * i + * 

y - y < |y  - y I implying  y - y <0  that  is. 

y (x,n)  > y (x,ft)  if  y (x,fi)  < x . 
n n n 

If  y = x we  must  show  y = x . Assume  y < x . Then,  by 
5.3,  either 

y , [x  + y ,W(n)]  < x + y 
n— x 

or 

y [x  - y*,L(II) ] < x - y , 
n— x 

or  both. 

In  the  first  case 

yn_itx  + y*,W(fi)]  < yn-1[x  + y*,W(JI)] 


and  in  the  second  case 
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y^lx  - y >L(n)]  < y^lx  - y ,L(n)]  . 

In  either  case  5.4  can  be  rewritten  as 

y+  - y < yjy^lx  + y+,w(n)]  - yn-1[x  + y ,w(fi)]  + 
yn_x[x  - y+,L(12) ] - yn_x[x  - y*,L(fi)]j 
+ * 

and  the  same  argument  shows  that  y < y . Since  we  had  assumed 
+ * 

y = x , y < x , this  is  a contradiction.  Hence  we  must  have 

y*  = x . This  shows  that  y (x,ft)  = x implies  y (x,II)  * x 

n n 

completing  the  induction.  ■ 
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APPENDIX 

SUPERADDITIVE  FUNCTIONS  AND  ISOTONICITY  THEOREM 

The  results  in  this  appendix  are  not  original.  They  were 
taught  to  the  author,  during  a course  in  inventory  theory,  by 
Donald  Topkis,  and  are  included  here  for  the  sake  of  completeness, 
since  no  reference  could  be  found  in  an  exhaustive  literature 
search. 

We  shall  be  concerned  with  the  following  problem.  Let 

f(x,y)  : S-*-R  where  x t En  y £ ^ , S C En  * if  . 

Consider  the  problem  max  f(x,y)  where  the  maximization  is 

x 

carried  out  over  x : (x,y)  e S . Let  x(y)  be  the  vector  x 
where  the  maximum  is  attained. 


r 
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> j,  7|  |t<  7 - 


¥ 


Al.l. 

*1 

E , the  complete  space 

A1.2. 

0 , the  null  set 

A1.3. 

any  subset  of  E^ 

Al  .4 . 

{x  c En  : x <_  b} 

A1.5. 

Zn  , the  set  of  all 

n-tuples  of 

rationals 

A1.6. 

\ x t En  : Xx . - yx . 
< i 1 

<_  ctj>  if  X 

, u 1 0 . 

The  proofs  that  these  are  sublattices  are  trivial. 


Lemma  Al: 


If  S is  a sublattice  Vy  £ r , then  so  is  OS 


I 


Proof : 

Elementary.  ■ 

If  S is  a sublattice  of  ED  k e”  , then  for  each  y e if 
define 


Sy  = {x  c E : (x,y)  e S}  the  section  of  S at  y 
nyS  = |y  e E111  : y4  0 j-  the  projection  of  S on  the  y axis 


Lemma  AL : 


S and  II  S are  sublattices. 

y y 


Proof : 


Easy. 


r 
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Definition  A2: 

A function  f : S-^R  is  superadditive  on  the  sublattice 
S C En  if  for  all  x^  , x^  e S 

f(x^  v x^)  + f(x^  a x^)  f(x1)  + f(x^)  • 

Theorem  A3: 

f : S— ►R  is  superadditive  on  the  sublattice  S C En  if  and 
only  if  for  all  i f (x  + e y^)  - f(x)  is  increasing  in  x^ 
j 4 i , where  y^  represents  the  i-th  unit  vector. 

Proof : 

Suppose  f is  superadditive.  We  have  to  show,  for  t , 6 > 0 
f(x  + e - f(x)  < f(x  + e ^ + 6yj)  - f (x  + 6y^)  j ^ i . 

It  is  clear  that 

(x  + e y^)  a (x  + 6y^.)  = x , (x  + e y^  v (x  + 6yJ  = x + t yi  + 6y^  . 
By  superadditivity 

f (x  + e yi)  + f(x  + <Sy  j ) <_ 
f ( (x  + e y a (x  + 6yJ)  + f + e v (x  + 

f (x  + e y + f (x  + 6y  ^ ) <_  f(x)  + f (x  + e + 6y^ ) 

which  establishes  the  necessity. 


r 
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Conversely,  pick  x , y t S . If  x = y then 

f(x  v y)  + f(x  A y)  = f(x)  + f(y)  . 

If  there  exists  an  i such  that  xi  ^ E Yj  Vj  # i then 

f (x  v y)  + f (x  A y)  = f (x)  + f (y)  . 

Now  assume  there  are  two  or  more  indices  i such  that  x^  i*  y^  . 
Assume  we  have  relabeled  the  indices  so  that  x^  t y^  , 

*2  i y2  •••  *p  * yp  . *p+1  - yp+1  y — *,  - • 

Consider  p = 2 . 

There  are  four  cases 


a) 

X1  - yl 

x2  -y2 

Then  x 

A y = x 

X V y = y 

and  f (x  a y)  + f (x  v y)  = f(x)  + f(y) 

b) 

Xl^l 

x2  -y2 

. A similar  argument  hold 

c) 

X1  -yl 

x2  < y2 

Then  x^  = y^  + e y2  = x2  + ^ ‘ The  condition  states 

f (y  + e - f (y)  >_  f (y  + e y1  - 6y2)  - f (y  - 6y2) 
f (x  V y)  — f(y)  >_  f (x)  ~ f(x  A y)  . 

Since 

x = y + e - 6y2 
x a y = y — 6p2 
xAy  = y + ep  . 
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d)  <_  x2  — ^2  ' ^ similar  proof  holds. 

For  p > 2 we  apply  repeatedly  the  same  argument,  increasing  step 
by  step  along  each  coordinate.  ■ 

Examples  of  superadditive  functions 

A2.1.  Any  real  function  on  E^ 

A2.2.  The  sum  of  superadditive  functions 

A2.3.  A positive  constant  times  a superadditive  function 

n 

A2.4.  A separable  function  f(x)  * £ f . (x  ) 

1 1 

A2.5.  If  f is  concave  on  E^  then  g(x^,x2)  * f (x^  - x2> 
is  superadditive  in  the  pair  because 

+ e,x2)  - g(x1x2)  - f(xx  + e - x2)  - f^  - x2) 
increasing  in  x2  by  the  concavity  of  f . The 
converse  is  also  true. 

The  proof  of  these  examples  is  trivial. 

Lemma  A4: 

Let  f be  superadditive  on  a sublattice  S C En  and  let 

* * 

S = {x  e S : f(,x)  >_  f(y)  Vy  e S}  . Then  S is  a sublattice. 

Proof : 

12*  * 12  12 
Pick  x , x e S . Since  SCS  xVxeS  x a x e S . 

1 2 

By  superadditivity  of  f and  optimality  of  x and  x 
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Thus  f(x^  v x2)  * f (x1)  « f (x2)  = f (x^  a x2)  the  maximal  value 

1 2 * 1 2 * 
of  f , implying  x vx  e S ,x  ax  eS  . ■ 


Theorem  A5: 


Assume  S is  a nonempty  compact  sublattice  of  E . Then 
S has  a greatest  element  x and  a smallest  element  x . 


Proof : 


We  have  to  show  that  there  exist  x and  x such  that 


Vx  e S x v x = x xAX  = x. 


For  i = 1,2,  ....  n pick  x1  e En  to  be  the  solution  to 


subject  to  x e S . 


x'  exist  since  we  are  maximizing  a continuous  function  over  a 

nonempty  compact  set  S . 

_ n , - ~i 

Pick  x = v x , Clearly  x e S since  x e S Vi 

i-1 

Vxt  S ,Vi  so  x v x « x Vx  e S . A similar 
construction  with  minimization  instead  of  maximization  yields  x 


Corollary  A6: 


If  the  superadditive  function  f is  continuous  and  the  set 
S is  compact  then  the  set  S of  Lemma  A4  is  nonempty  and  hence 
has  a greatest  and  least  element. 
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Theorem  A7 : 

Let  S C En  x e"  be  a sublattice.  For  x e En  , y e Em  . 
Assume  f(x,y)  : S— *-R  is  superadditive  on  S . Then 

g (y)  = max  f (x,y) 
xeS 

y 


is  superadditive  on  IIyS  . 


Proof : 


1 

y 


Pick  y1  , y2  e IlyS  . 
2 

and  y respectively. 


1 2 

Let  x and  x be  optimal  for 
Then 


gCy1)  + g(y2)  = f(x1,y1)  + f(x2,y2) 

<_  f(x^  A X2,y^  A y^)  + f(x^  V X2,y^  V y^) 

£ gCy1  A y2)  + gCy1  v y2) 


Definition  A3: 

Let  A and  B be  sublattices  of  En  . We  say  B dominates 
A (A  ^ B)  if  whenever  a e A , b e B we  have  a a b e A , 
a v b e B . 

Lemma  A8: 

The  set  of  *>ii  nonempty  sublattices  of  En  is  partially 
ordered  by  the  relation  ^ . 


Proof : 


We  need  Co  show 

a)  A ^ A trivially  true 

b)  A ^ B and  B ^ A implies  A = B . 

Choose  b e B . Then  for  a e A a a b e A because  A * B . 

But  b e B thus  bv  (a  a b)  eA  because  B ^ A . But 
b v (a  a b)  = b so  b e A . 

c)  A ^ B B ^ C implies  A ^ C . 

By  enumeration  of  cases  one  can  verify  that  for  a e A , b e B 
ceC  aVc=[(bAc)va]vc.  Then  b a c e B (B  ^ C)  so 

(b  a c)  v a e B (A  ~<  B) 


so 


a v c = [ (b  v c)  v a]  v c e C (B  ^ C)  . 

Similarly  by  noting  that  a a c = [ (a  v b)  a c]  a a we  can  show 
a a c e A . ■ 

Definition  A4: 

If  T C E™  is  partially  ordered  by  some  relation  C we 
say  St  C En  is  ascending  in  t on  T if  t C t'  implies 


a 
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Lemma  A9: 


n m 

If  S is  a sublattice  of  E * E then  is  ascending 

in  y e E™  on  II^S  under  the  usual  vector  ordering. 


Proof : 


so 


Pick  y^  , e n^S  <_  y^  . 

Pick  e S ^ e S 2 • Then  (x\y*)  e S , (x^,y^)  e S 

y y 


1212  121  12 
(x  Ax,y  Ay)eS=*>(x  Ax,y)eS^>x  ax  eS 


111")  "\  9 9 19 

(x  vx,y  v yz)  e S =*>  (x  vx,y)eS=*>x  vx  £ S ^ 


Lemma  A1Q; 

If  A and  B are  sublattices  of  En  with  A ^ B , and  both 
A and  B have  greatest  (least)  elements  a , b (a,b)  respectively, 
then  a <_  b (a  b)  . 

Proof : 

Since  A ^ B we  have  a v b e B implying  a v b <_  b since 
b is  the  greatest  element  of  B . But  b <_  a v b by  the 
definition  of  the  v operator.  Thus  a <_  a v b * b . Similarly 
aAbeA=^aAb>a  but  aAb<a  so  a = aAb<b.B 


We  now  come  to  the  main  result  in  this  appendix. 


J 
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Theorem  All:  Isotonicity  Theorem 

Let  f be  superadditive  on  a sublattice  S C En  x E10 


n in  * 

For  x e E , y e E let  be  the  set  of  all  optimal  x for 


the  problem  max  {f(x,y)  : x e S } . 

* * 

Then  S is  ascending  in  y on  y : S f 0 

y y 


Proof : 


1 2 12*.-* 

Pick  y and  y such  that  y <_y  S ^ J , S ^ l1  0 . 


* * y y 

We  will  show  S ^ j S ^ . 

y y 

1 * 2 * 11 
Choose  x e S ^ , x e S 2 • Clearly  (x  ,y  ) e S 

y y 


(x2,y2)  e S . Also 


(x1 , y1 ) a (x2,y2)  = (x1  A x^y1) 


(x^y1)  v (x2,y2)  = (x1  v x2,y2) 


Now,  since  f is  superadditive,  and  from  the  optimality  of  x , 
2 


121  11  22  122 
0 > f(xX  a x ,y;  - f^.y1)  > f (x%y*)  - f(xX  v > 0 


Thus 


1 A 2 1. 

f(x  a x ,y  ) 


1 XN 

f(x  ,y  ) 


max  f (x , y ) 
xeS  , 


1 2 * 1 2 * 
implying  x a x e S ^ . Similarly  x v x £ S j 

y y 


1 


1 


f 


hi 
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As  a consequence  of  this  theorem  and  Lemma  A10  we  have 
12  * * 

immediately  that  if  y y and  both  S ^ and  S 2 have  greatest 

y y 

_1  —21  2 

(least)  elements  x and  x (x  and  x ) respectively,  then 
-1-2  1 2 

x <_  x (x  <_  x ) . Furthermore,  even  if  such  elements  do  not 

12  12 

exist  we  can  find  ordered  solutions,  since  x a x <_  x v x and 

1 2 * 1 2 * 
x Ax  e S i x vx  eS2. 

y y 

Corollary  A12: 

Suppose  S is  a compact  sublattice  for  all  y and  f(x,y) 

* 

is  continuous  in  x for  all  y e TI^S  . Then  is  a nonempty 

compact  sublattice  for  all  y . They  have  greatest  and  least 
elements  which  are  isotonic  on  II  S . 

y 

Remark 

It  is  clear  that  if  our  problem  is  one  of  minimization,  we  can 
consider  the  negative  of  the  function.  Equivalently  we  may  define 
subadditive  functions  in  the  obvious  manner,  and  results  similar 
to  the  above  will  hold. 

To  show  that  x(y)  is  decreasing  in  y , we  can  proceed  to 
redefine  our  problem  in  terms  of  v = - y and  show  x(v)  is 
increasing  in  v . Thus  endless  variations  of  this  theme  are 
possible,  but  they  can  all  be  trivially  reduced  to  the  case  discussed 
in  this  appendix. 


